Large-deviation principles, stochastic effective actions, path entropies, and the 
structure and meaning of thermodynamic descriptions 
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The meaning of thermodynamic descriptions is found in large-deviations scaling of the prob- 
abilities for fluctuations of averaged quantities. The central function expressing large-deviations 
scaling is the entropy, which is the basis for both fluctuation theorems and for characterizing the 
thermodynamic interactions of systems. Freidlin-Wentzell theory [3j provides a quite general for- 
mulation of large-deviations scaling for non-equilibrium stochastic processes, through a remarkable 
representation in terms of a Hamiltonian dynamical system. A number of related methods now 
exist to construct the Freidlin-Wentzell Hamiltonian for many kinds of stochastic processes; one 
method due to Doi 0,0] and Peliti 0, Qi appropriate to integer counting statistics, is widely used 
in reaction-diffusion theory. 

Using these tools together with a path-entropy method due to Jaynes [^, this review shows 
how to construct entropy functions that both express large-deviations scaling of fluctuations, and 
describe system-environment interactions, for discrete stochastic processes either at or away from 
equilibrium. A collection of variational methods familiar within quantum field theory, but less 
commonly applied to the Doi-Peliti construction, is used to define a "stochastic effective action", 
which is the large-deviations rate function for arbitrary non-equilibrium paths. 

We show how common principles of entropy maximization, applied to different ensembles of states 
or of histories, lead to different entropy functions and different sets of thermodynamic state variables. 
Yet the relations of among all these levels of description may be constructed explicitly and under- 
stood in terms of information conditions. Although the example systems considered are limited, 
they are meant to provide a self-contained introduction to methods that may be used to systemat- 
ically construct descriptions with all the features familiar from equilibrium thermodynamics, for a 
much wider range of systems describable by stochastic processes. 



I. INTRODUCTION 

Thermodynamics is not fundamentally a theory of en- 
ergy distribution, but a theory of statistical degener- 
acy . As such, while most of our experience and intu- 
ition about thermodynamics is drawn from equilibrium 
statistical mechanics (lol - [T2| . which emphasizes the role 
of energy, we should expect that its fundamental princi- 
ples apply in much wider domains, outside equilibrium, 
and even outside mechanics. 

Within the last 50 years, a clear conceptual un- 
derstanding of the nature of thermodynamic descrip- 
tions [ll, [H has combined with new methods to analyze a 
wide variety of stochastic processes, for continuous [ isj 
and discrete systems , in quantum [3, [^] and clas- 
sical mechanics [l^, [TJ , both at and away from equilib- 
rium, and even in other areas using information theory 
such as optimal data compression and reliable commu- 
nication llSll . and via these, robust molecular recogni- 
tion [l^ [20|. These developments confirm that thermo- 
dynamics is indeed not restricted to equilibrium or to me- 
chanics. They give us insight into when thermodynamic 
descriptions should exist, and they provide systematic 
methods to construct such descriptions in a wide variety 
of situations. 

This paper reviews the aspects of large-numbers scal- 
ing and structural decomposition that are essential to 
thermodynamic descriptions, and presents examples from 
which each of these may be seen in different forms that 



are appropriate to equilibrium and non-equilibrium sta- 
tistical mechanics. It also brings together construction 
methods (based on generating functions W47|), scaling 
relations (based on ray approximations 0, |21||). and vari- 
ational methods (based on functional Legendre trans- 
forms [in ) , which will enable the reader to systematically 
construct the fluctuation theorems of thermodynamic de- 
scriptions from their underlying stochastic processes. 



A. Key concepts, and source of examples 

1. Entropy underlies large-deviations scaling and reflects 
system structure 

The entropy, as a logarithmic measure of degeneracy, 
is the central quantity in thermodynamics. It arises as 
the leading term in the log-probability for fluctuations 
of averaged quantities. At the same time, however, when 
sub-system components, or a system and its environment, 
interact, they discover their most probable joint config- 
uration through fluctuations. Therefore the competition 
among entropy terms also reflects the structure of system 
interactions at the macroscale. We are reminded of the 
importance of this structural role of entropies, by the fact 
that pure "classical" thermodynamics is entirely de- 
voted to the analysis of entropy gradients. Therefore we 
wish to insist on being able to decompose entropy into 
its sub-system components as a criterion for any fully- 
developed thermodynamic description. 
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The key property that defines the existence of thermo- 
dynamic limits, and the form of their fluctuation theo- 
rems, is large-deviations scaling [H, It is the precise 
statement of the simplification provided by the law of 
large numbers, not only in the infinite limit of aggre- 
gation, but in the asymptotic approach to infinity. It 
is well-known that averaging over ensembles of configu- 
rations for large systems removes almost all (irrelevant) 
degrees of freedom, and leaves only summary statistics, 
which are the state variables of the thermodynamic de- 
scription.^ In finite systems, these summary statistics 
can still show sample fluctuation, but in the thermody- 
namic limit, the fluctuation probability takes a simple 
form. 

For an ensemble that possesses large-deviations scal- 
ing, it is possible to describe classes of fluctuations as 
having the same structure under different degrees of ag- 
gregation. (An example would be fractional density fluc- 
tuations in regions of a gas, whose sample volume may be 
varied). The log-probability for any such fluctuation then 
factors, into a term that depends on overall system scale, 
and a scale-invariant coefficient that depends only on the 
structure of the fluctuation. The scale-invariant coeffi- 
cient is called the rate function of the large-deviations 
scaling relation (An example would be the specific 
entropy of a gas.) 

Large-deviations scaling presumes the existence of a 
separation of scales - between microscopic processes and 
their macroscopic descriptions - over which aggregation 
does not lead to qualitative changes in the kinds of fluctu- 
ations that can occur. We expect thermodynamic limits 
to exist where this separation of scales is large. Examples 
of structural change that can interrupt simple scaling un- 
der the law of large numbers include phase transitions, 
which can change the space of accessible excitations. 

Large-deviations scaling permits us to combine fluc- 
tuation statistics with entropy decompositions that re- 
flect system structure. In equilibrium thermodynam- 
ics, the result is the classical fluctuation theorem for 
macrostates the log-probabflity for fluctuations (of 
energy, volume, particles, etc.) between sub-systems 
with well-defined entropies is the difference between the 
sum of entropies at the fluctuating value and the maxi- 
mum value for this sum, which is the equilibrium value. 

If we wish to consider the probabilities of fluctuations 
with more complex structure (whether in equilibrium 
thermodynamics or in more complicated cases), we will 
need more flexible methods to compute large-deviations 
formulae and entropy decompositions. For this, we intro- 
duce the notion of the stochastic effective action, which is 



In the language of Kolmogorov, the state variables are minimum 
sufficient statistics for predicting the state of a system sampled 
from an ensemble. No more detailed summary statistic provides 
better predictions over the whole ensemble. At the same time, 
no further reduction makes it possible to write any state variable 
as a function of the others. 



deflned by variational methods that generalize the famil- 
iar Legendre transform of equilibrium thermodynamics. 
The meaning of this quantity, and the way it is used, will 
be most easily understood by following its construction in 
the body of the paper, so we postpone further discussion 
until that point. 



2. The principles in a simple progression: from equilibrium 
to non- equilibrium statistical mechanics 

We will develop examples whose purpose is to show 
that the deflnition and properties of the entropy do not 
change as we extend thermodynamics beyond equilib- 
rium statistical mechanics. Rather, what changes is the 
state space to which we assign probabilities. 

A direct example comes from comparing an equilib- 
rium thermodynamic system, to a non-equilibrium de- 
scription constructed for the same system. A frequent 
approach (23l - [26| to non-equilibrium statistical mechan- 
ics (NESM) continues to use the equilibrium state vari- 
ables and equilibrium entropy, but considers their time 
rates of change. We will see that such an approach, fo- 
cused on retaining the functional form of the equilibrium 
entropy, sacriflces its meaning as a large-deviations rate 
function. 

Instead, we will make the transition from equilibrium 
to non-equilibrium statistical mechanics by replacing an 
ensemble of states (in equilibrium) by an ensemble in 
which entire time-dependent trajectories - termed histo- 
ries - are the elementary entities (for NESM), and then 
we will construct the appropriate large-deviations limits 
for the ensemble of histories. The functional form of the 
entropy will necessarily change. More importantly, the 
inventory of state variables will necessarily be enlarged, 
to include not only the conflguration variables of equilib- 
rium, but also a collection of currents that relate to the 
changes in configuration. Both kinds of variables will be 
needed as summary statistics for an ensemble of histories, 
and both will enter as arguments in the non-equilibrium 
entropy. 

NESM is not so far removed from equilibrium thermo- 
dynamics that it can really do justice to the generality of 
large-deviations principles and thermodynamic descrip- 
tions. However, it allows us to begin with a completely 
familiar (equilibrium) construction, then to compare it 
to a construction with rather different functional forms, 
and finally to derive the complete set of relations that 
connect the two descriptions. 



B. Markovian stochastic processes, the two-state 
model as an example, and the approach of the paper 

Markovian stochastic processes provide a general, 
substrate-independent framework within which to study 
statistical degeneracy and large-deviations scaling. They 
include models from statistical mechanics, but they may 
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also be used to represent many other random processes, 
whose structure may have different constraints and inter- 
pretations from those of mechanics. 

This review will use the two-state random walk, in ei- 
ther discrete or continuous time, as a sample system for 
which equilibrium and non-equilibrium thermodynamic 
descriptions will be built and then compared. The two- 
state random walk is the simplest discrete stochastic pro- 
cess, and most thermodynamic quantities of interest in 
both ensembles can be computed for this system in closed 
form. However, the constructions in the examples imme- 
diately generalize to more complicated cases, and several 
generalizations and approximation methods will be cov- 
ered either in the main text or in appendices. 

Sec. [TT] will introduce large-deviations scaling one level 
"below" the discrete random walk, by supposing that the 
discrete model emerges as a coarse-grained description 
from the continuous random walk in a double-well po- 
tential. The continuum model illustrates the concept 
of concentration of measure from large-deviations the- 
ory, and sets the parameters (both explicit and implicit) 
that define the discrete model. It also clarifies the nature 
and origin of the "local-equilibrium" approximation [2^ 
for coarse-grained descriptions of motion on free-energy 
landscapes, and illustrates graphically the dual roles that 
charges and currents must have in non-equilibrium en- 
tropy principles. 

Sections IIIII and IIVI present the equilibrium thermody- 
namics of the two-state model, and its most direct gen- 
eralization through the master equation of the stochastic 
process. Sec. |IlT] uses the exact solution of the equilib- 
rium distribution to introduce all basic quantities of the 
large-deviations theory, and derives these using gener- 
ating functions and their associated variational methods. 
Sec. lIVI then presents the same construction for the time- 
dependent probability distribution, in which histories of 
particle counts rather than counts at a single time are 
the elementary entities. Neither of these ensembles dis- 
tinguishes particle identities, either in states or in tra- 
jectories. Sec. |V] presents an alternative construction of 
a thermodynamics of histories based on the entropy of 
distinguishable-particle trajectories, and this second con- 
struction naturally separates the path entropy from prob- 
ability terms due to the environment, in a form exactly 
analogous to the Gibbs free energy for equilibrium. 

The remainder of the introduction lists sources for 
the particular methods used in later sections, and ex- 
plains why they capture different aspects of a full non- 
equilibrium thermodynamics. Many aspects of the fol- 
lowing derivations - the naturalness of the generating- 
function representation, the role of operator algebras and 
linear algebra more generally, or the information condi- 
tions and counting statistics that relate one ensemble to 
another - may be understood in conceptual terms that 
are more fundamental than the particular constructions 
in which they appear below. We return to these in the 
discussion, making use of examples from the text. 

Numerous, diverse literatures now contribute to the 



understanding of methods closely related to those used 
below. A brief summary of the history and connections 
among the ideas, and a broader set of citations, are pro- 
vided in App. |X1 



C. Bringing together three perspectives on 
non-equilibrium thermodynamics 

The extension of large-deviations scaling to ensembles 
of histories is given by Freidlin-Wentzell (F-W) the- 
ory 3 . This approach is widely applied to the computa- 
tion of escape trajectories and first-passage times ^2S-t3(^] . 
It is remarkable for the way it reduces both problems of 
inference, and the description of multiscale dynamics, to 
a representation which is a Hamiltonian dynamical sys- 
tem [Ullli. 

A convenient method for arriving at the Hamiltonian 
description of F-W theory is the Doi-Peliti (D-P) con- 
struction [4-7] based on generating functionals. The D- 
P construction is only one of many closely-related meth- 
ods based on expansions in coherent states, which are re- 
viewed in App. El These methods Ll^,,21,.3^^] have the 
common feature that the field representing sample-mean 
values of observables is augmented by a conjugate mo- 
mentum that generates the change in those observables. 
This value/momentum pair leads to the F-W Hamil- 
tonian description. The D-P method is particular to 
stochastic processes with independent, discrete number 
counts, but may readily be generalized to continuous or 
non-independent observables [4(| , as well as having many 
parallels in dissipative quantum mechanics [l^. HE , [41 1 . 

The F-W method directly gives the scale factors and 
rate functions of the large-deviations limit for histories. 
However, it does not generally decompose fluctuation 
probabilities into separate terms representing sub-system 
components or system-environment interactions, which 
we want as part of a thermodynamic description. To pro- 
duce that decomposition, we introduce a path-entropy 
method due to Jaynes known as mELximum caliber 
which has its roots in much older analysis of the en- 
tropy rates of stochastic processes and chaotic dynam- 
ical systems due to Kolmogorov and others. In addition 
to making the F-W representation more recognizable as 
a direct counterpart to the constructions in equilibrium 
thermodynamics, the maximum-caliber method will sep- 
arate those events that involve energy exchange with the 
environment from those that do not, allowing us to un- 
derstand the role of energy dissipation in large-deviations 
formulae for paths. 



D. A glossary of notation 

In characterizing thermodynamic ensembles, we will 
need to make three choices about the level of representa- 
tion, and the basis used: 
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1. Whether we are referring to ensemble averages 
(hence, deterministic summary statistics), or quan- 
tities that fluctuate stochastically to represent the 
process of sampling; 

2. Whether we are considering discrete samples that 
change in the integer basis of particle counts, or 
modes of collective fluctuation, which we describe 
with the continuous mean values of Poisson distri- 
butions;^ 

3. Whether we are describing absolute particle num- 
bers distributed among states, or are separating to- 
tal number as a scaling variable, from the fractional 
distributions of particles that have a scale-invariant 

This distinction is similar to the distinction between the position 
basis and the wavenumber basis exchanged by Fourier-Laplace 



meaning. 

Different choices will lead to objects with quite differ- 
ent mathematical behavior, even if all of them represent 
particle numbers in one way or another. The notation in 
the following sections is chosen to reflect the three choices 
above, while still permitting readable equations. 

Ensemble averages of any quantities are denoted by 
overbars. This convention is easier to incorporate in 
complicated equations than the E (for expectation value) 
commonly used in statistics. 

The remaining distinctions in the notation are summa- 
rized in Table 

transform. 



Position 


Domain Measure Dynamics Meaning and usage 


Conjugate 


variable 






momentum 


N 


[0,oo] 


Integer Fixed Total population number 


not used 






Scale factor in large-deviations property 




Hi 


[0,N] 


Integer Stochastic Values of sample points 


—d/dui 


Ui 


[0,N] 


Real Langevin Mean of Poisson distribution 




Ui = Ui/N 


[0,1] 


Rational Stochastic Relative values of sample points 


- (1/iV) d/dn. 


Vi = Ui /N 


[0,1] 


Real Langevin Distribution normalized by 








Structure factor in large-deviations property 





TABLE I: Notations for particle number in different bases, n^ are integer sample values of particle counts in state i. rii are 
the corresponding continuous mean values of Poisson distributions used as a basis for collective fluctuations, fii are integer 
sample values normalized by total number to isolate large-deviations scaling, and Vi are the corresponding normalized mean 
values of Poisson distributions. Each representation of population numbers is accompanied by a conjugate momentum or shift 
operator, shown in the last column. Total population A'^ will be kept fixed in all examples, and its corresponding shift operator 
is therefore not used. 



It will be important, in following the Doi-Peliti con- 
struction below, to understand that math-italic are 
the mean values of Poisson distributions that are used as 
a basis in which to expand the actual distribution that 
evolves under the stochastic process. They remain ran- 
dom variables that are sampled from the ensemble, but 
they fluctuate with Langevin statistics rather than the 
Poisson statistics of the integer particle counts n^. The 
ensemble mean will be denoted fii. 

The dynamics of a stochastic process may be repre- 
sented in three ways associated with these different bases, 
which contain equivalent information and which will all 
be illustrated in the following sections. These are: 1) the 
transfer matrix of the discrete stochastic process, which 
shifts probability among indices; 2) the Liouville opera- 
tor that acts on generating functions, shifting probability 
among modes; and 3) the action functional of the Doi- 
Peliti field-theoretic expansion in Poisson distributions, 
which generates the covariance for Langevin statistics. 
We will show that, with a suitable choice of dynami- 



cal variables, one may skip over the laborious task of 
interconverting these representations, and simply copy 
the functional forms from one representation to another. 
The number variables and shift operators in Tabic II Dl 
are those that substitute for one another under changes 
of representation. 



II. LARGE-DEVIATIONS SCALING AND THE 
SEPARATION OF SCALES: CONCENTRATION 

OF MEASURE; THE SKELETONS FOR 
EQUILIBRIUM VERSUS DYNAMICS; NATURE 
OF THE LOCAL-EQUILIBRIUM 
APPROXIMATION; EXISTENCE OF A 
NATURAL SCALE 

In later sections, the two-state random walk will be the 
microscopic model, whose thermodynamic descriptions 
we seek. In this section we will take an even lower-level 
random walk in a continuum potential to be the micro- 
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scopic model, for which the two-state model is the coarse- 
grained thermodynamic description. Starting in the con- 
tinuum will allow us to relate the dual (charge/current) 
character of non-equilibrium thermodynamic state vari- 
ables to graphical features of the underlying free-energy 
landscape. The particular class of continuum models we 
will consider - landscapes with basins and barriers - also 
lead to the asymmetry between states and kinetics in 
classical discrete stochastic processes, and to the nature 
of the local-equilibrium approximation in such models.^ 
The main points of the section will be: 1) the way 
large-deviations scaling isolates the fixed points of a 
stochastic dynamical system as its control points; 2) the 
difference between the relevant sets of fixed points for 
equilibrium versus non-equilibrium ensembles; and 3) the 
way the static and kinetic properties of the underlying 
system become encoded in the discrete representation. 
We will also cover the origin of the natural scale for a 
discrete stochastic process, which is never expressed di- 
rectly within the discrete model, but which can be neces- 
sary to regulate and to understand divergences when its 
stochastic behavior is analyzed. 



stable states and transitions. We will see that the mo- 
mentum variables that appear in the Freidlin-Wentzell 
theory are precisely the distortions of free energy land- 
scapes that extract the past histories most likely to lead 
to fluctuation conditions imposed at any moment. 



B. Concentration of measure in the 
large-deviations limit, and fixed points on the 
free-energy landscape 

Consider then the stochastic motion of a particle in 
the double- well continuum potential illustrated in Fi g. [I] 
Large-deviations theory for the continuum 0, 
applies at low temperatures, and characterizes three as- 
pects of the probability of trapped motions and escapes: 




A. The free-energy landscape representation 
bridges scales, and applies to general stochastic 
processes satisfying detailed balance 

Entropy characterizes probabilities for systems that we 
describe completely (called "closed" systems), while free 
energies characterize probabilities for systems coupled to 
incompletely-described environments. A review of the 
basic relations of probability to entropy and free energy in 
classical equilibrium thermodynamics is given in App. |B] 

In this and later sections, we will write probabilities 
of states, and transition rates, in terms of Gibbs free en- 



ergies The free energy representation is not linked 



to any particular scale, and so provides the map from a 
continuum potential for a stochastic particle to the pa- 
rameters of its discrete-state approximation. This rep- 
resentation becomes particularly powerful for extensions 
from single-particle motion, to the conversion of several 
species of particles in fixed proportion, which occurs in 
chemical reaction networks (considered in App. |D]). The 
origin of free energies need not be mechanical; such a 
representation may be found for any stochastic process 
that satisfies detailed balance in its stationary state. 

Free energies will also provide a way to interpret the 
generating functions and functionals used in later sec- 
tions. A generating function distorts the free energy 
landscape by changing the one-particle free energies of 



^ This asymmetry is not inherent in the classical limit, however, 
nor is it a common feature of quantum statistical ensembles, 
where positions and momenta may have much more symmetric 
roles. The continuum model therefore provides a point of depar- 
ture for several, different macroscopic approximations. 



FIG. 1: A continuum double-well potential that might under- 
lie the two-state model. The fine-grained description follows 
particle trajectories over the continuum. The coarse-grained 
description is dominated by three chemical potentials: at the 
minima of the well (attracting fixed points) and the barrier 
maximum (the one-dimensional version of a saddle point). 



1. The probability for a random walker to be found 
away from the attracting fixed points (the minima 
of the continuum free-energy potential) decreases 
exponentially in /? — l/fc^T, where T is tempera- 
ture and ks is Boltzmann's constant. In particu- 
lar, escapes from one well to the other are expo- 
nentially improbable. The leading term in the log 
of the escape rate is given by the quasipotential of 
Freidlin-Wentzell theory [H]. 

2. Among escape events, the majority occur along a 
particular spatio-temporal history of thermal exci- 
tations leading uphill from the minima of the po- 
tential toward the interior maximum. More pre- 
cisely, conditional on having observed an escape 
event, the probability that the escape trajectory 
deviated from this most-probable form decreases 
exponentially in /3 — l/ZcsT, which is a large- 
deviations result for histories. The most-likely es- 
cape trajectory is the stationary path derived from 
the Freidlin-Wentzell dynamical system [2^. For 
one-dimensional systems, this trajectory is always 
the time-reverse of the classical diffusion trajectory 
from the maximum to the minimum. 

3. The most-probable trajectory for any escape re- 
quires a specific finite time, which will limit the 
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frequencies at which we can use coarse-grained ap- 
proximations. In one dimension, the escape time 
equals the classical diffusive relaxation time over 
the path whose time-reverse is the escape. 

The large-deviations scaling parameter in these contin- 
uum formulae is the inverse temperature /3 = l/ZcsT. /3 
will continue to be a scaling parameter in large-deviations 
formulae for the discrete approximation, but other pa- 
rameters such as the number of random walkers in the 
system will also enter. 

The exponential suppression of deviations from either 
fixed points or stereotypical escape trajectories is the 
phenomenon known as concentration of measure for the 
random walk in the potential. It is the property that 
leads us to make only a finite error^ even if we replace 
the infinitely many degrees of freedom of a random walk 
on the continuum, by a (zero-dimensional!) discrete ran- 
dom walk between two states, written 

b¥^a. (1) 

Here b and a are coarse-grained labels standing for the 
left-hand and right-hand wells. 

The states in the discrete walk are associated approx- 
imately with the positions of the local minima of the 
continuum potential, as shown in Fig. [51 Each state also 
has a Gibbs free energy per particle - known in chemi- 
cal thermodynamics as the chemical potential [01, and 
written (for a single particle) as fi^ - near the value at 
the well minimum. 




FIG. 2: Discrete hops between well minima account for al- 
most all structure in the probability distribution for the con- 
tinuous random walk at low temperature. The role of the 
barrier potential (saddle point in Fig. [T|) as the determiner of 
kinetics is absorbed in the transition rates k± . 



The continuum large-deviations theory for escapes [28[ 
gives the escape rate from either well, to leading expo- 
nential order, as a function of the chemical potentials in 
the wells and at the local maximum of the continuum 
potential between the wells (indicated in Fig. [1]) , in the 
form 

= e-^i^t-'^i). (2) 



Note that the absolute magnitudes of probabilities which 
are kept to describe transitions are generally (exponen- 
tially) smaller than corrections to the mean properties of 
the fixed points, which are omitted in the large-deviations 
approximation.'^ They are, however, the leading terms 
in the conditional probability, and therefore the leading 
contribution to dynamics. 

The escape rates appear in the discrete-state, 
continuous-time approximation as the parameters in its 
master equation, whose form is 



~d/dn, _ i) n 



+ fc_(e^/^"''-^/^""-l)nJpn„^„,. (3) 



dt 



Here, n^ and n^ are possible values for the numbers of 
particles found in the right and left wells at any instant 
of time. Pn„.n5 is a time-dependent probability density 
for an ensemble of such observations. The master equa- 
tion ([3]) describes the fiow of probability among different 
values of the indices (n£i,nf,) as particles hop with rates 
k± per particle. 

Often the right-hand side of a master equation such as 
Eq. ([3]) is written as a sum over all values of (ua, n?,). For 
stochastic processes in which almost all change results 
from independent single-particle transitions, only adja- 
cent values (ua ± 1 , Ub =p 1 ) contribute to the change in 



Therefore we have written the shifts of indices in 



the sum as the action of operators ^d/dnt-d/dna ^ treating 
the indices (na,nb) formally as if they lived on a con- 
tinuum, even though only values separated by integers 
ever appear in the time evolution of Pn„,n[,- The opera- 
tor in square brackets in Eq. ([3]) is called the transfer 
matrix of the stochastic process. Its functional form will 
re-appear throughout the subsequent analyses, in the op- 
erators that govern the time evolution of generating func- 
tions, and as the Hamiltonian in the Freidlin-Wentzell 
dynamical system. 



C. The discrete-state projections for equilibrium 
versus non-equilibrium systems 

The net effect of concentration of measure in the con- 
tinuum random walk is to extract the parameters of its 
discrete approximation from the fixed points of the mean 
fiow on the free-energy landscape. The concentration to- 
ward the attracting fixed points is purely spatial, and is 
easy to visualize for landscapes in any number of dimen- 
sions. The concentration of escape trajectories is spatio- 
temporal, and is not directly illustrated in the static po- 
tential of Fig. [T] Concentration of trajectories takes on 
a spatial aspect for landscapes in more than one dimen- 



The magnitude of this error decays as powers of l/kgT. 



^ See Ref. O . Ch. 7 for more on approximations of this kind. 
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sion, where transitions are exponentially likely to pass 
through saddle points, as shown in Fig. O 




FIG. 3: A Free energy landscape in more than one dimen- 
sion, and its approximation by a discrete-state system. The 
charge- valued state variables of the equilibrium ensemble live 
at the attracting fixed points (black), where the principle axes 
of curvature (crosses) point in the same direction, so the scalar 
curvature of the landscape is positive. Current-valued state 
variables of the kinetic ensemble live on the saddle points 
(white), where principle axes of curvature point in opposite 
directions, and the scalar curvature is negative. Trajectories 
mostly sit at fixed points for long time intervals, but when 
transitions happen, they are exponentially likely to follow spe- 
cific escape paths between basins, indicated by narrow curved 
paths. 

We may now graphically characterize the difference be- 
tween static and dynamic ensembles, for random walks 
on landscapes with basins and barriers. The equilibrium 
thermodynamic description for any such system is fully 
specified by the chemical potentials fi^ at the attracting 
fixed points (filled dots in Fig. [3]) . In this ensemble, there 
is no role for barriers, and no appearance of their chem- 
ical potentials fj,^. The probabilities for state occupancy 
are determined only by their free energies, because all 
waiting times for escapes are (by assumption) surpassed. 

For systems away from equilibrium, transition rates be- 
come essential to determining state occupancies, as well 
as transition frequencies between pairs of states. These 
rates are controlled by the saddle points (white dots in 
Fig.©. 

With each kind of fixed point we associate a state vari- 
able in the thermodynamic description. (See App. iBjfor 
discussion of the origin and role of state variables as con- 
straints.) The state variables of the equilibrium theory, 
which live on attracting fixed points, all have the prop- 
erty of charges: their value would not change if we ran the 
dynamics in time-reverse. Away from equilibrium, new 
state variables are introduced, which live on the saddle 
points, and these have the property of currents: their 
value would change sign if we ran the dynamics in time- 
reverse. 

Non-equilibrium ensembles require the introduction of 
additional sets of current-valued state variables [4^, , 



which do not arise in equilibrium, and which have their 
origin in properties of the saddle points of the underlying 
free-energy landscape. 

D. The nature of the local-equilibrium 
approximation 

Free energy landscapes with basins and barriers lead 
to an extreme asymmetry in the way charge- valued and 
current- valued state variables are represented in the dis- 
crete model. The asymmetry comes, as noted above, 
from the fact that the leading probabilities for dynamics 
are exponentially smaller than corrections to the prop- 
erties of states that are dropped in the large-deviations 
approximation. Therefore, in such systems, the charge- 
valued state variables, even in the non-equilibrium en- 
semble, are nearly identical to the state variables of equi- 
librium. The one-particle chemical potentials /i'^, and 
their generalizations to concentration-dependent chemi- 
cal potentials all have the same relation to Gibbs 
free energy as in equilibrium. This property defines the 
local- equilibrium approximation for this class of models. 

We note two things about the local-equilibrium ap- 
proximation, to emphasize its limits. First, the equiva- 
lence of the non-equilibrium charge- valued state variables 
with their counterparts in equilibrium does not extend to 
the entropy (43| . The non-equilibrium entropy depends 
on both charges and currents, even at a single time. For 
ensembles of histories, it depends on the rate of tran- 
sitions as well as the state-occupancy statistics, as we 
will show in multiple examples in Sec. IIVI and Sec. |Vl 
This fact will be essential to understanding "maximum- 
entropy production" [2^ |43 - |47| as an approximation but 
not a principle for non-equilibrium systems. 

Second, we note that the asymmetry between states 
and kinetics need not be a property of free energy land- 
scapes if they do not have basins and barriers. In par- 
ticular, it may not be a property of free diffusion, and it 
is generally not a property of driven dissipative quantum 
ensembles (43| . Therefore, if the underlying continuum 
model does not have the features that create asymmetry, 
we have no ground to expect that even the charge- valued 
state variables in the non-equilibrium theory will closely 
resemble those in the corresponding equilibrium limit. 



E. The implicit "natural scale" for a 
coarse-grained description 

Finally we mention an implicit limit on the use of the 
discrete-state approximation. Escapes in the continuum 
model are rare within the intervals that particles spend 
in cither basin. However, they do require a non-zero 
time, comparable to the diffusive relaxation time. In 
the non-equilibrium discrete-state model, we will probe 
transition probabilities with time-dependent distortions 
of the chemical potentials /i^ and The model will per- 
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mit these probes to have arbitrarily high-frequency time- 
dependence. However, we will see when we consider path 
entropies in Sec. |V] that such sources lead to divergences 
in individual entropy terms that should remain finite to 
be meaningful. 

The solution to the problem of divergences is to rec- 
ognize that the discrete model has a natural scale [48| . 
which is an upper bound on the frequencies that may 
sensibly appear in probes of the theory. The natural 
scale is the diffusive relaxation frequency in the con- 
tinuum model. For probes with higher frequencies, the 
constants k± describing transition rates no longer retain 
their meaning or values. They were defined as lumped- 
parameter representations of escape paths, in a potential 
which was assumed to be fixed during the period of the 
escape. Faster probes "melt" the discrete approximation, 
and require that the description revert to the underlying 
continuum. 



III. BASIC QUANTITIES INTRODUCED 
WITHIN THE EQUILIBRIUM ENSEMBLE: 

GENERATING FUNCTIONS AND THE 
EXPRESSIONS OF LARGE-DEVIATIONS 
SCALING 

We now begin the analysis of the equilibrium distribu- 
tion for the two-state system. Since the entire equilib- 
rium distribution may be written down analytically (it 
is a binomial distribution), the purpose of this "analy- 
sis" is to introduce the key quantities expressing large- 
deviations scaling, along with systematic ways to com- 
pute them using generating functions. The methods 
and the asymptotics will generalize immediately to time- 
dependent systems that are not exactly solvable, or at 
least very inconvenient to write in closed form. 

The large-deviation result we will derive is that, for any 
apportionment (na,nh) of N particles to the two states, 
the log-probability of this apportionment in the equilib- 
rium distribution is the difi^erence of the joint entropy 
from its maximizing value. A more extensive taxon- 
omy of large-deviation results for equilibrium ensembles 
is given in Ref. Non-maximizing values of (n^, Uf,) are 
termed fluctuations, and the relation between entropies 
and log-probability for sample values is therefore called 
a fluctuation theorem. 

We will isolate this leading-exponential term in the 
log-probability by using the cumulant-generating func- 
tion to shift the distribution, effectively projecting onto 
a sub-distribution within which (n^, n;,) is the most-likely 
value. The sub-distribution, when normalized, would be 
the equilibrium distribution in a two-state system with a 
shifted chemical potential; the ratio between the gener- 
ating function and the normalized distribution measures 
the overlap of the original distribution with the one ap- 
propriate to (na,nb). 

It will be the Legendre transform of this cumulant- 
generating function that gives the fluctuation probabil- 



ity to observe (na,n;,) in the original equilibrium dis- 
tribution, and expresses this probability as a difference 
of entropies. The Legendre transform of the cumulant- 
generating function is known, in some domains of quan- 
tum field theory, as the quantum effective action, and we 
will adapt that term here, calling it the "stochastic ef- 
fective action" . (For more context and the relation to 
literature, see App. \^) Though it is only a difference of 
static entropies in the equilibrium ensemble, the stochas- 
tic effective action will become dynamical in ensembles 
of histories, where it will be the strict counterpart to the 
quantum effective action. 



A. The equilibrium distribution of the two-state 
stochastic process 

At an equilibrium steady state the solution to the mas- 
ter equation ^ is the binomial distribution 




(4) 



Total particle number N = Ua + nt, is conserved by all 
terms in the transfer matrix of Eq. ([3]). The equilibrium 
occupation fractions are 



in which 



k+ + k- Zi 

Z, = e-^^» + e-^^^ 



(5) 



(6) 



is the called the partition function for a one-particle 
ensemble in this two-state system. 

The Gibbs free energies for non-interacting particles 
scale linearly (that is, they are "extensive") in parti- 
cle number so the structural terms in the large- 
deviations formulae for fiuctuation probabilities will be 
functions of the ratios 



nh 
N' 



(7) 



In both steady-state and time-dependent probability dis- 
tributions, Roman Uq, n?,, fia, h;, will be used for dis- 
crete particle-number indices, respectively un-normalized 
or normalized by A^. When the description of distribu- 
tions is transferred to the generating function, the corre- 
sponding continuous indices will be n^, n{, for absolute 
number, and Va, Vb for relative numbers corresponding 
to the definitions ([7]). 
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For dynamical as for static systems, it is convenient 
to study open-system properties by considering an open 
system and its environment to be components in a larger 
closed system. Here the closed system will be defined 
by iV as a parameter, and we introduce the stochastic 
variable under the reaction n = (n^ — Ua) /2, so that 



N 
1 
N 



(8) 



When only n is denoted explicitly, the distribution Pn^.nt 
will be indexed pn- The relative particle number asym- 
metry is likewise defined as n = (hf, — iio) /2. Its counter- 
part in continuous variables will be denoted v. Equilib- 
rium values for all numbers will be indicated with over- 
bars. 

The equilibrium distribution ((4]) may be cast in a va- 
riety of instructive forms. Using the relation ([5]) of the 
rate constants to one-particle chemical potentials, and 
Stirling's formula for the logarithms of factorials, the fol- 
lowing expressions re equivalent: 

1 



V27rlVn^ 
1 

y/2TTNnanb 
1 



(log n„ -log iia ) +115 (log -log ) 



V2^iVh,hb 

1 



g-JVZ5(ri||fi) 

g-^Ila (^fJ,l +T log iia) -^Ili, {l^l+T log flfc) 



1 



V27r7Vn^ 



gJVlog(Af/Zi)g-/3n<.Ma 



gjVlog(jV/Zi) ^ / Qg ^-p(G^+G,) 

27rnanh 



(9) 



In the second line, £'(fi || fi) is the Kullback-Leibler di- 
vergence pgj . in which fi and h stand for the distribu- 
tions with coefficients (ha,fifc), (ha, fib) respectively. The 
Uq- and nft-particle chemical potentials add concentration 
corrections to the entropy in the one-particle potentials, 
as 



Ma 
Mb 



Mi 
Mfc 



fesTlogUa 

/csTlog ni,. 



(10) 



By comparing the second with the last-two lines of 
Eq. we see that the minimum of the Gibbs free en- 
ergies of the subsystems over n is 

N 

P min[Ga + Gb\ = P min[naMa + n&Mfo] = ^ log t;-- 

n Zi 

(11) 

The one-particle minimum expressed in terms of frac- 
tional occupancies, which is the descaled version at any 
N , gives an iV-independent relation between the chemi- 
cal potential per particle, and the one-particle partition 
function, 

niin[na (mI + /^sTlogiia) + fih {yl + fcsTloghb)] 
= -fcsTlogZi. (12) 



These are the standard relations for ideal gases or dilute 
solutions. 



1. The aggregation of state variables and the fluctuation 
probability 

The local-equilibrium approximation for the two-state 
system allows us to approximate the log-probabilities for 
non-equilibrium configurations of (na,nf,) by the sum of 
free energies for the individual wells. At the minimizing 
value of (uq, n;,) for this sum, the equilibrium free energy 
for the composite system is attained. We may therefore 
express the minimum joint free energy per particle, using 
Eq. ([TT|). as a definition for the single-particle chemical 
potential for the equilibrated system, in a form equivalent 
to to Eq. (Uni): 

min[Ga + Gb] = Maub = -fcsT log + /jbT log N 

= Maufc + fcisTlogAT. (13) 

Here Mau& whole-system counterpart to the one- 

particle chemical potentials m^ and Mb for subsystems in 
the local-equilibrium approximation. 

The density values in the equilibrium distribu- 
tion dH) may then be written as exponentials of the 
(system + environment) entropies of Eq. (IBSp , as 



27rnanb 



(14) 

27rnQnb 

As explained in App. [B] all three Gibbs free energies are 
functions of intensive fc^T and p, and of extensive argu- 
ments Ua, Ub, and N. Since total energy is controlled by 
the environment temperature, it is the entropy compo- 
nents of these G values for the subsystems, as functions 
of Ha and nb, which control the difference of Ga + Gb from 

GaUb- 



2. Entropies of equilibrium and residual fluctuations 

The leading-exponential approximation of large- 
deviations scaling separates extensive entropies of the 
subsystems, which were defined into the parameters of 
the two-state stochastic process from coarse-graining the 
continuum model, from entropies due to chemical fluc- 
tuation, which are sub-extensive. To see this, following 
any standard thermodynamics text 



11| . we write any of 



the one-particle chemical potentials m^ 
which is the specific enthalpy and is the specific 
entropy. This decomposition yields a relation between 
subsystem and whole system free energies at equilibrium, 
which is 



'aUb 



- fib/lb 

fibSb - ha log ha - fib log fib. (15) 
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The free energy in the thermodynamic limit is then given 
by 



and the associated one-particle partition function with 
source q is 



Gaub = N (/i^ub - fcsTsJub + fcsTlog AT) , 



(16) 



(«) 



(21) 



which extensive in particle number, if we think of the 
log N term as being set by pressure. In comparison, the 
Shannon entropy of the full distribution over fluctuations 
contains a term from the normalization of the exponential 
in Eq. (ITU) , due to its width. 



The sum in the third line of Eq. (|19p evaluates to unity, 
as for any normalized binomial, but it is instructive to 
use what was learned in forming Eq. (|14|) to recast the 
sum and prefactor together in terms of a normalized dis- 
tribution and a "penalty" term, as 



p7 \ogpl^ « i [1 + log (2^7Vfi^h,)] . (17) V(z) « e^G—'^Gi^^ V /^^L±i^,-K«^"+G<'") 



This correction, being only logarithmic, is sub-extensive 
in N. 



B. Generating functions and the stochastic 
effective action 



A moment- generating function - or "ordinary power- 
series generating function" (50| - for a distribution in- 
dexed on the two numbers n^ and n;, has two complex 
arguments, and is written 



(18) 



To study the properties of pn at fixed A'', recognizing that 

•2""^"° = {zaZb)^^'^{zb/ ZaY" , wc may set ZaZb = 1 and 
denote z^j z^ = z. At equilibrium we will be interested 
only in the one-argument function of z. However, as we 
pass to dynamical descriptions, it will remain convenient 
in some cases to retain both variables Za and z;,, even if 
they are applied to a distribution with support on only 
one value of N . 

The weight factors z" have an effect similar to shift 
in the subsystem chemical potentials, which will recur 
repeatedly in our analysis. Therefore we denote z = e*, 
and write the one- variable generating function as 

n 

= ^ (na/\/i)"°(h,,yi) 




(19) 



Here new normalized fractions in the presence of q 
which will be referred to as a source ~ are defined by 



-g/2 



hae-«/2 + n{,e9/2 



nf, 



n&e 



q/2 



=-(/3Mi-g/2) 



hae-«/2 + hbe9/2 z[ 



i1) 



(20) 



^TTnaUb 

(22) 

In Eq. ([2^ the subsystem free energies are defined at any 
lia, Ufe as 

Gi') = Ga+nakBTq/2 



Gi"^ = Gb-nbkBlq/2. 



(23) 



We wish to introduce g[^^^ as the minimum of G^'' +g[''' 
over n, as before. However, for discrete n, this gives a dis- 
continuous function of q. The thermodynamic usage of 
^aAjb ^ macroscopic function of its intensive state 

variables, and therefore it will save intermediate steps 
and notation simply to define G^^j, as the minimum over 
n treated as a continuous variable, whose minimizing ar- 
gument will then be a continuous function of q. 

For such binomial distributions or their multinomial 
generalizations, sources of the form kslq = q/ (3 behave 
as shifts in chemical potential, in this case split evenly 
between subsystems a and b. The "penalty" term is ex- 
pressed as a function of (^Gaub — G^aubj ' minimum 
chemical work needed to convert the system with poten- 
tial difference pb — Pa and equilibrium fi to one with po- 
tential difference Pb ^ Pa ^ q/ P and an equilibrium h^'"' 
determined by the new potentials. P {Cavjb — G^^y^^ is 
also the log-probability to obtain the shifted distribution 
from the original through the set of weights z". 

The penalty function is referred to as the cumulant- 
generating function and denoted T{q). It is defined from 
the moment-generating function by the relation 



^(z) = e 



r(g) 



(24) 



From the definition (|19p of the moment-generating func- 
tion it follows that 



dlog V'(z) 
dlog z 



dq 



(25) 



in which we introduce the continuous counterpart 71(9) as 
both the gradient of F and the expectation of n under 
the equilibrium distribution shifted by z. 

From Eq. ([H]), the cumulant-generating function is 
just the Gibbs free energy difference 



r(9) = PGi'^b - PGaUb 



(26) 
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Recasting Eq. ((25 



that 



dT{q) _ dG_ 
dq d{—kB'Tq) 



(9) 

aUb 



(27) 



we recover the usual relation from equilibrium thermo- 
dynamics [ll| , that the gradient of the Gibbs free energy 
with respect to the chemical potential is the particle num- 
ber. 

For reference in later sections, we may compute closed 
forms for the various quantities. Define p = ^l^, dual 
to the relative particle-number difference h. Then 



and 



r(g) - N 



,M ^^tanh^iq-m, (28) 



log cosh ^ — log cosh —{q — I3fi) . (29) 



In continuum field theories with many particles and 
nonlinear interactions among them, it is often necessary 
to approximate the moment- and cumulant-generating 
functions by series expansions in the variance of the 
Gaussian approximation to fluctuations. In such an ex- 
pansion. The gradient of F yields all of the connected 
graphs giving n^'^\ while the gradient of ip gives a sum 
over all graphs (see Ref. [2^] Ch. 16). 



1. The stochastic effective action for single-time 
fluctuations 

App |B] reviews the fact that the Gibbs free energy is 
a Legendre transform of the entropy. Thus, the entropy 
is the converse Legendre transform of the Gibbs free en- 
ergy. We may therefore expect that by Legendre trans- 
forming T(q), we will arrive at a direct expression for the 
entropy differences that govern internal fluctuations of 
particle number, without reference to external tempera- 
ture or pressure. 

The Legendre transform of the cumulant generating 
function F is known as an effective action |22| . When 
it is computed for the single-time binomial distribution, 
it hardly seems to justify its name, if we expect an ac- 
tion in the sense of Hamiltonian dynamics. However, it 
will become just such an action in the time-dependent 
ensemble. Here we introduce the Legendre transform for 
its statistical meaning, and later we introduce the dy- 
namical version. 

The definition is 



[r(<z)+nq]|^^^<„,. 



(30) 



in which q^") denotes the inverse function of n'-''' from 
Eq. ([271). ^ From the gradient relation it follows 



dn 



(31) 



Therefore S'^^ must vanish at the mean value of n in the 
distribution p for which it is computed, because no source 
(i.e., q{n) = 0) is required to yield the equilibrium value 
as the mean. 

Addition of the term to F in Eq. (l30l) cancels only 
the explicit q/jS term in the shifted free energy G^^'J^j, leav- 
ing the particle assignments ni'\n[''^ unchanged. The 
terms that remain are precisely those in the fluctuation 
theorem: they are the subsystem free energies evaluated 
at shifted ns. The effective action may be written 



(32) 



In Eq. (15^ the free energies G are now regarded as func- 
tions of the continuous- valued and Uf, rather than the 
discrete indices Uq and n^ on which the distribution is 
defined. 

Referring back to the distribution over fluctuations in 
the original p^'J „ of Eq. IH)) . we see that exp(— 5'°*'') 
directly extracts the leading exponential dependence of 
the fluctuation probability. It omits sub-extensive nor- 
malization factors related to the width of the distribu- 
tion. Therefore the effective action directly expresses the 
large-deviations scaling for fluctuation probabilities. The 
log-probability approximated by Eq. ([5^ scales linearly 
in N, and its structure is determined by the remaining 
A^-invariant fractions Va and i^b- 

For reference in comparison to later expressions for the 
effective actions of fluctuating histories, the effective ac- 
tion for the equilibrium distribution evaluates to 



N 



log cosh ^ — log cosh - ^(7^" 



/3fl 



(n) 



■ tanh — 
2 



/3/2 



(33) 



in which g*^") is evaluated by inversion of Eq. 



IV. ENSEMBLE OF HISTORIES I: THE 
MASTER EQUATION FOR NUMBER; 
DOI-PELITI CONSTRUCTION OF THE 
GENERATING FUNCTIONAL; THE 
DYNAMICAL-SYSTEM REPRESENTATION OF 
FREIDLIN AND WENTZELL 

We now construct a second generating function and 
ensemble, for the time-dependent distribution p evolving 



For the equilibrium distribution, and indeed all other distribu- 



tions computed below, this inverse will be unique. For problems 
in which it is not unique, such as occur along the co-existence 
curves of first-order phase transitions, the Legendre transform is 
replaced by the Legendre- Fenchel transform 0, 0] . 
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under the master equation ([3]) , rather than merely for its 
equihbrium steady state. The equihbrium fluctuation re- 
sults of Sec. Illll will reappear within this larger construc- 
tion, as probabilities for histories conditioned on their 
behavior at a single instant but on no other information. 
Here, however, the single-time fluctuation probabilities 
will no longer be computed as primitive quantities. In- 
stead, they will arise as integrals of log-probability along 
the entire histories which are most-likely, conditioned on 
achieving a particular deviation at a single time. 

In keeping with the shift from states to histories, 
the cumulant-generating function and effective action in 
this section will be functionafe, whose arguments are se- 
quences of occupation numbers (nQ,nf,). The description 
derived from the master equation will still only count 
particle numbers, and will not resolve identities. Un- 
like the equilibrium constructions, the path-space large- 
deviations formulae will not allow us to identify the nat- 
ural path-entropy functions, for which we will require the 
caliber formulation of Sec. |Vl However, this section will 
introduce the remarkable Freidlin-Wentzell formulation 
of large-deviations theory as a Hamiltonian dynamical 
system, in which the stochastic effective action will be- 
come a genuine Lagrange-Hamilton action functional.^ 



A. Liouville time evolution and the coherent-state 
expansion for its quadrature 

The reason to study generating functions at equilib- 
rium is that the moments of the equilibrium distribution 
may be more relevant and robust descriptions of fluctua- 
tions than individual probability values Pna,ni,- Similarly, 
the reason to use generating functions to study dynamics 
is that the basis of single-particle hops is often less in- 
formative than a basis in the collective motions of many 
particles. 

The master equation formalism acts on probabilities 
in the basis of single-particle hops. The counterpart, 
which acts on the moment-generating function, is known 
as a Liouville equation. When specified in complete de- 
tail, the Liouville equation has the same information 
as its corresponding master equation. However, low- 
order approximations to the Liouville form will often 
contain more of the total information than similarly 
reduced approximations in the basis of single-particle 
hops. Such approximations are sometimes called the hy- 
drodynamic limit |52| . Therefore the Liouville form is 
the starting point for methods of Gaussian integration, 
Langevin equations, or other commonly used approxima- 
tion schemes. 



For an introduction to action functionals and Hamiltonian dy- 
namics, see Ref. [sil . 



1. The master equation expressed as a function of 
mass-action rates 

The two-state master equation ([3]) is an instance of 
a wider class of equations describing single-particle ex- 
changes. A notation for the more general class is intro- 
duced here, both because it cleanly divides the particle- 
hopping terms associated with stochasticity from the 
probabilities which coincide with mass-action rates, and 
because it immediately generalizes to more complex reac- 
tion stoichiometrics, such as those required by chemical 
reaction networks. The master equation for the two-state 
system is written in this general form as 

^= E (e^/^"'-^/^"-l)^..Ps. (34) 

i,je{a,b} 

Here n is a vector whose components give the number of 
particles of each type i G (here ii = (ua, Uf,)). For 

the linear reaction model, the mass-action rates rji^ 

rta fc+Ua 

Tab = fc-Ub (35) 

describe the probability for particle transfer from state 
i to state j. The form ([M]) extends immediately to ar- 
bitrary many-particle exchanges with rates that may in- 
volve powers of many different components of n, and to 
arbitrary network connections. The general methods de- 
rived below extend immediately to all those cases. Even 
the shortcuts we will introduce, to pass directly from the 
form of the master equation to the action functional of 
Freidlin-Wentzell theory, apply with no changes.® 

2. Definition of the Liouville operator from the transfer 
matrix 

This section sketches the conversion from number 
terms and index shifts in the master equation, to the 
equivalent polynomials and differential operators in the 
Liouville equation. It then converts the representation on 



We write the mass-action rates as going from i to j with this 
index ordering, because they will appear again later as entries in 
a transfer matrix, which is naturally written as acting from the 
left on column vectors of occupation numbers [n^] 
The reader is cautioned that in cases where multiple reactants of 
the same type participate in a reaction, the assumption of sepa- 
ration of timescales may be inconsistent with using equilibrium 
state variables in the minimal way they are used here. An exam- 
ple is provided in Ref. [ssll . where the local-equilibrium approx- 
imation may still be used, but it requires replacing the discrete 
master equation with a full Boltzmann transport equation for 
continuous-valued positions and momenta. An alternative solu- 
tion is to add time-local corrections known as "contact terms" 
to represent the correct counting of many-particle states. For 
the linear system such complications do not arise, so we will not 
digress on them further here. 
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polynomials to the representation introduced by Masao 
Doi in terms of raising and lowering operators in 

an abstract linear vector space. The systematic construc- 
tion of Liouville forms from master equations is now stan- 
dard [l^, [13] , and its form for the linear mass-action ki- 
netics of Eq. (|35l) is the simplest possible. 

In the vector notation, the generating function at a 
single time becomes a function of a vector z whose com- 
ponents correspond to those of fi. Here the moment- 
generating function is 



m-Y. n 



PS- 



(36) 



n i^{a,h} 



Time evolution under Eq. 
■0 according to 



34)) implies time evolution of 



We may efficiently approximate the product (|38)) by 
interposing a complete distribution of generating func- 
tions between each of the small increments of time evo- 
lution in the second line, which we develop in the next 
sub-section. Doing so, however, introduces a distinct set 
of arguments z at each time, which are cumbersome to 
work with. Therefore, at intermediate times between 
and T, it is conventional to adopt an abstract notation 
for the complex vectors and derivatives [l, Q , which more 
clearly reflect the nature of the generating function as a 
vector in a normed linear space (a Hilbert space). The 
correspondence of the elementary monomials. 



z. 
d_ 

dz, 



■3 ^ 



O a 



(39) 
(40) 



dt 



E n 4:>t 



i,je{a,b} 



n kG{a,b} 



d 



- E E n 



n kG{a,b} 



(37) 



Here we have written out the functional dependence of 
Tij on the number indices of the particle species enter- 
ing half-reactions explicitly, to show how those number 
indices are replaced by products Zid/dzi acting on the 
basis polynomials of the generating function. Successive 
lines of Eq. ((37)) reflect the action of successive terms in 
the master equation . The extension of these steps to 
multi-species reactions also follows a standard form ^1^, 
of which our example is representative. The differen- 
tial operator C{z^d / dz) is the Liouville operator for this 
stochastic process. 

Time evolution of the density pfi over an interval [0, T] 
is performed by taking the quadrature of Eq. ([37]), which 
we may think of as being performed over a sequence of 
short time intervals of length 5t: 



T/St 

fe=l 



(38) 



T denotes time- ordering in the product, and we will use 
the same notation when the product of exponential terms 
e"^*-^ is written as the exponential of a sum or integral. 
Such expressions are known as time- ordered exponential 
integrals. For the Liouville operator with constant co- 
efficients, time-ordering serves no explicit purpose, but 
when time-dependent sources are introduced below, it 
will become necessary. 



reflects the property that complex variables and their 
derivatives satisfy the algebra of raising and lowering op- 
erators familiar from the quantum harmonic oscillator,^" 



d_ 

dzi ' 



(41) 



Polynomials built from these variables live in a func- 
tion space built on a "vacuum" which is simply the num- 
ber 1, denoted 



lo|0). 



(42) 



The conjugate operation to creating a generating func- 
tion is the evaluation of the trace - generally with 
moment-sampling operators to extract its moments - 
so that the conjugate operator to the right-hand vac- 
uum (jl^ is an integral. 



d^zS{z) 



(43) 



From these definitions an entire function space with inner 
product can be built up, as reviewed in Ref's. [TgI. [T7|.^^ 



3. The coherent-state expansion 

The transformation from discrete number-states to 
generating functions is the Laplace transform. It may 



Of course, in the approach of "quantizing" the classical Har- 
monic oscillator to a differential operator on Schrodinger wave 
functions, this map from polynomials to operators is the origin 
of the raising and lowering operator algebra. 

It is conventional to denote the inner product for classical 
stochastic processes with round braces, as in JO), to distinguish 
them from the quantum bra-ket notation |0) |54| . This notation 
reflects the fact that it is not in the operator algebra, but in 
the space of functions and the inner product, that the distinc- 
tion between classical and quantum-mechanical systems lies, as 
discussed in the introduction. 
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be applied within the discrete time slices of the quadra- 
ture (PS)) . where it makes use of the Laplace transforms 
of intermediate Poisson distributions, which are known 
as coherent states. It has become conventional in the 
Doi-Peliti literature to denote the mean values of these 
Poisson distributions with the field cf); for a multi-state 
system, we index them with a complex- valued vector field 
(j). The conjugate vectors to these Poisson distributions 
are moment-sampling operators, weighted by Hcrmitian 
conjugate field variables denoted 0^. 

The 0^ will have the property that, dynamically, they 
propagate information backward in time from final data 
imposed by the weights Za, zt- The interpretation of this 
property is that the moment-sampling operators capture 
conditional distributions at earlier times, conditioned on 
the final values they produce. The fields (p ^-nd (f)'' will 
turn out to be the position fields and their conjugate- 
momentum fields in one version of the Freidlin-Wentzell 
theory, and the Hamiltonian under which they evolve 
will be the Liouville operator. The way evolution un- 
der the dynamical system is made consistent with the 
"backward" propagation of information, by the conju- 
gate momentum, is that the momentum variables will 
typically have unstable evolution forward in time, while 
the position variables will evolve in stable directions. 

In this section we construct the coherent-state expan- 
sion for the quadrature ([55)1 . which is due to Peliti [a,[3|- 

The Liouville operator is a function of raising and low- 
ering operators {a'}, so the convenient expansion 

of Eq. ([55)) will be given by a basis of eigenstates of these 
operators. The coherent states are constructed to be such 
eigenstates. The right-hand coherent state is defined as 



-(<?t.0)/2 ^ 



M=0 



mT 



M 



|0) 



|0).(44) 



The components of its complex parameter vector (j) corre- 
spond to those of z. From the commutator algebra (|4T1) , 
and the fact that all a* O d/dzi annihilate the right vac- 
uum (im, it follows that Eq. (|44l) does define an eigen- 



state of all the lowering operators a' with corresponding 
eigenvalues 



(45) 



by 



The left-hand coherent state dual to Eq. is given 



(^^1 = {0\e^'-'e-i^'-^)^-' 



(01 E 



M 



M=0 



M! 



(46) 



Eq. (pS)) is likewise checked to be an eigenstate of the 
raising operators, with eigenvalues 0^, 



(47) 



The presence of a corresponding to d/dz in Eq. p6)) 
gives the left-hand coherent state the interpretation of 
a moment-sampling operator. This state displaces any 
argument in the state that it acts on (through the in- 
ner product), by a summand It then sets the original 
a] to zero because all left-hand states include the trace 
defined by the left vacuum 

The sum of outer products of a left-hand coherent state 
and its dual right-hand coherent state gives a representa- 
tion of the identity operator as an over-complete integral 
over states. 



= I. 



(48) 



The result of taking the inner product of Eq. (|48p with 
any state ipj on the right is that the moment-sampling 

operator (^(j)^ simply transposes the weights that it ex- 
tracts from ipj onto the left-coherent states 
expansion PSI) . 

If we insert a complete set of states in this form be- 



the 



tween every term in the product (|38p . we express the 
moment-generating function at time T as an integral over 
all intermediate values of the field variables (f, (j)^ , of the 
form 



1p{Za, Zb) = e 



-Ft (log Za, log Zb) 



p4p0a2?0l;2?0be(^''-'^"T)'^"^+(^''-'^^T)'^''T~f^-ro(iog0t^,iog0tj^ ^^^^ 



It requires some algebra, but no conceptual difficulties, to show that the action in Eq (I49p is given by 

fdt\- (dt<l>Ua + dt4<l>b) + C] . (50) 
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The terms dt4'\4>a come from the inner products of states 

and 0^ at successive times, and the evaluation of 

the Liouville operator sandwiched between these states 
converts C into the quantity serving as the Hamiltonian. 
Its form, 



(^1 



(51) 



is obtained directly from the penultimate line of Eq. ([57)1 
with the mass-action rates (|35l) . It is a general feature, 
which extends to non-linear rate equations, that factors 
of Zi in the rate functions on the reactant side cancel 
factors \/zi from the shift operators (the leading paren- 
thesized term in Eq. ([57|) '). 



B. Standard computations of the Freidlin-Wentzell 
quasipotential 

Equations (|49ll5ip are the stochastic-process reformu- 
lation of the single-time generation function (|18p . The 
Legendre transform log iji^Za, Zb) continues to be the equi- 
librium effective action S'^^{na,nb) of Eq. ([50]) . The 
field integral (|49)) for these quantities will in general be 
dominated by the stationary paths of the action (|5(I1) . 
The value of S in Eq. ([SO]) , on the stationary path, is 
known in Freidlin-Wentzell theory as the quasipotential, 
because it generalizes the Gibbs free energy from equi- 
librium thermodynamics.^^ The quasipotential is widely 
used to approximate the solutions to diffusion equations 
on the boundaries of trapping domains 0, [2l1 . l38j . 

The field integral of the last section has converted an ir- 
reversible stochastic process into a deterministic dynam- 
ical system perturbed by fluctuations. Any such Hamil- 
tonian system offers several choices of representation, 
related under the changes of (field) variables known in 
Hamiltonian dynamics as canonical transformations [5]| . 
In addition to the canonical variables of the Hamiltonian 
dynamics, this action functional also admits an unusual 
"kinematic" interpretation, in which escape trajectories 
are represented as rolling in an energy potential, even 
when they move opposite to the direction of classical dif- 
fusion.^"^ Finally, the leading expansion about the mean 



behavior in the integral (l49l) comes from Gaussian fluctu- 
ations, which are readily converted to standard approx- 
imation methods such as the Langevin formulation. For 
the bilinear action, the Langevin approximation in the 
original fields (f>, (f>^ is exact, though in other variables or 
for more complex reactions it will not be. We will con- 
sider each of these aspects of the Doi-Peliti field integral, 



The Gibbs free energy and other Legendre transforms of the en- 
tropy are conventionally known as thermodynamic potentials, by 
yet a further analogy with the potential energy in mechanics. 
A thorough treatment of this kinematic potential is provided in 
Ref. [H. 



and the Freidlin-Wentzell quasipotential, in turn in this 
section. 

All introductions to Freidlin-Wentzell theory that I 
have seen in the reaction-diffusion literature compute the 
quasipotential by studying the stationary points in the 
field variables^c/)''' i'^ equivalent operator 

expectations [331 ■ ^'^^ the two-state system, this choice 
is technically the easiest, because the bilinear functional 
integral is manifestly the "simple harmonic oscil- 
lator" of stochastic processes. However, field variables 
are the least intuitive choice, because individually nei- 
ther (j) nor 0^ represents an observable quantity. (Recall 
that the function of as the expectation value of a Pois- 
son distribution is only propagated through time by the 
moment-sampling weight given by An elementary 

canonical transformation to action-angle variables [5l| 
will produce fields which, unlike 4> and (j>\ correspond 
directly to observed average number occupancies, and to 
a conjugate momentum that has the physical interpreta- 
tion of a chemical potential. 



1. Diagonalization and descaling in field variables 

The quasipotential provides the leading log-probability 
for fluctuations. When large-deviations scaling holds, 
the value of the quasipotential on its stationary path 
should factor into an overall scale factor, and a scale- 
independent rate function. This scaling is particularly 
easy to achieve for the the Gaussian functional inte- 
gral of the two-state model. The field variable cj) 
is rescaled to remove a factor N, which may be moved 
outside the entire action ([50]). leaving a bilinear func- 
tional of normalized fields that do not depend on system 
scale, at any field values. In standard treatments using 
coherent-state variables [l^, [s^l, all scale factors come 
from the Poisson field 0, and their Hamiltonian-conjugate 
momenta 0^ are unchanged. 

Field rescaling is one of several changes of variable that 
may be made in the bilinear field basis given by (j), and 
<f>^ . Another is a rotation of components from {(j)a,4'b) 
to a basis in which the independent dynamical quan- 
tity n = (rib — no) /I may be separated from the non- 
dynamical conserved quantity N . However, this rotation 
must be used with care, as certain terms needed to define 
correlation functions in terms of the equilibrium distri- 
bution disappear from the nai've continuous-time limit, 
and must be reconstructed from the explicit discrete-time 
forms SQj. We will also see that correct calculation of 
the cumulant-generating functional can rely on contri- 
butions from the apparently non-dynamical number N , 
depending on how total derivatives are handled in the 
action ([50]) . 

In order to work out the correct treatment of such tech- 
nical issues, we will begin in this section with the gener- 
ating function if{za,Zb) of Eq. (jl8p . with its apparently 
superfluous second complex argument. Once the two- 
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variable generating function has been understood, we will 
perform the more complicated, but more direct, evalua- 
tion of the single-argument form ■ip{z) from Eq. (fT9| . 

We will see in this section that the coherent-state vari- 
ables (j), and (j)^ provide a powerful and simple route to 
scaling and a variety of exact solutions in the Gaussian 
model. However, they are not directly related to ob- 
servable quantities, and their simplicity in the case of 
non-interacting particles can quickly give way to quite 
complicated correlation functions if particle interactions 
must be considered. They also produce a continuum limit 
which, although simple in form, does not explicitly rep- 
resent all quantities needed to compute correlation func- 
tions. Therefore we will abandon the field basis of </>, and 
(j)^ as soon as the structure of its generating function has 
been understood, and move to a set of transformed vari- 
ables that lead to slightly more complex algebra, but do 
not suffer from any of these shortcomings. 

The field components (j)a, 4>b correspond to expectation 
values Ua, defined as in Eq. ([25)) .-^'^ Therefore, as 
for the equilibrium distribution, we may define a rotated 
basis corresponding in a similar manner to n and N , by 
the orthogonal transformation 



0t ^ 
$ = 
<i>t = 



{4>b " 4>a) /2 



4>i+^i) /2. 



(52) 



A scale factor proportional to total particle number may 
then be removed from the Poisson fields 0, $ only, defin- 
ing normalized fields 



(53) 



Here the relation of absolute to relative number fields 
repeats the notation used to relate absolute to relative 
number indices n and fi = ti/N . Even though the relation 
between variables 0, 0^ and the expected number indices 
is not simple, in general it is the 4) variables that carry 
the scaling with total system size. 

A final coordinate transformation reflects the fact that 
the coordinate timescale dt is not characteristic of the 
dynamics for either average behavior or fluctuations. We 
therefore introduce a rescaled time variable r, with Ja- 
cobean 



-^(Ait-piul,) 



ZlVaVb 



(54) 



For the case 0t = i representing thermal equiUbrium and all 
classical diffusion solutions, (f)a = na and 1/15 = nj, as an expec- 
tation in the functional integral, but not as a general insertion 
in higher-order correlation functions. 



as the physically relevant time variable. Note that this 
natural timescale for rare events relates the transition- 
state chemical potential to that for the equilibrated joint 
system, defined from Eq. (|13p . 

In these new variables the action (l50l) becomes 



S = N dT 



N j dT (-a^$t$_5^^t^ + £~ 



(55) 



2. Calculation of the two -argument cumulant- generating 
function 

The leading large-TV exponential dependence of the 
functional integral comes from the stationary point 
of the action and the boundary terms at times and T. 
The stationary-point conditions are vanishing of the first 
variational derivatives of the action ([55]). which take the 
form 



dr(f> 



d£ _ 
dC 
dC _ 

dt 



(56) 



Eq. (|56p is the promised conversion of the first-moment 
dynamics of the stochastic process (p4|) into a deter- 
ministic dynamical system with Hamiltonian C. It fol- 
lows immediately from these equations and from the 
lack of explicit time-dependence in C, that this Liouville- 
Hamiltonian is a constant of the motion along the sta- 
tionary path. 



dC 
d^ 



0. 



(57) 



The boundary terms at time T = T (fc_|_ + fc_ ) in the 
variation of the functional integral vanish at (j>'lf = 

Za, 4>bf = Zb- The intermediate time-dependence of (p^ is 
thus 



= {Zb - Za) 

and the corresponding solution for "I>t is 



(58) 



$t = - (zb + za) + i> (zfc - Zb) (1 - e-^) . (59) 

The field $ is non-dynamical, and a steady-state so- 
lution for the relative number field </> in terms of $ is 
therefore give by 



(60) 
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The product (j)l4>a + <pl4'b is the expectation of the total 
number operator, and therefore must equal N. In the 
rotated and descaled basis, this equality takes the form 



1 



- {Zb + Za)+l' {Zb 



Za) 



(61) 



which may therefore be used to assign a value to in 
terms of the arguments z^, Zb of the generating function. 

The equivalent expression for the relative number- 
operator difference (^(l)\(j)b — ^l^a^ /N is then 



2v = + 2$^<^. 



(62) 



Eq. ()62|) is immediately solved from the boundary val- 
ues (|58|59p at time f to recover Eq. as 



2vt — tanh - (log z — /3/j,) . 



(63) 



In terms of this final value for the function the func- 
tional dependence at earlier times is given by 



{i>T ~ v) 



(64) 



The algebra of the preceding solution is simple and lin- 
ear, and yet it is profoundly obscure as a representation of 
the physical distributions of random walkers in the two- 
state model. This obscurity is the reason we will soon 
abandon coherent-state variables. Note that the field </) 
- naively corresponding to the observable n - is actu- 
ally constant by Eq. (|60p . All particle number-dynamics 
in this solution comes from the fields (j)^ and $t These 
fields, known as response fields in the Doi-Peliti litera- 
ture [39[, are acting to propagate information from the 
boundary condition at the final time T backward into the 
interior of the functional integral, by selecting moments 
of the intermediate states with time-dependent weights. 

The stationary-path solution (|64p is the least- 
improbable sequence of fluctuations to have led to the 
final value Vr- The single-time probability of Vr in turn 
results from the accumulation of probabilities for succes- 
sive accumulating fluctuations in the integral (|55p for S. 
However, to identify the particle distribution in this solu- 
tion requires incorporating the response fields, including 
the (strangely dynamical) response field associated 
with the (unchanging) total particle number N . 

To show that, despite the difficulties of interpretation, 
the quasipotential recovers the single-time fluctuation 
theorems, we evaluate the cumulant-generating function 
Tf. The calculation is easiest if we take T ^ 1 so that the 
initial condition Fq may be evaluated on the equilibrium 
distribution.^^ For the exact bilinear action ([501 a con- 
servation law greatly simplifles the calculations, although 



Any other initial condition would decay toward the equilibrium 
distribution for sufficiently large T. 



it again renders the correct answer in a most cryptic fash- 
ion. 

We begin with the conservation law ([57]) . To obtain 
a reference value for C we note that in the equilibrium 
distribution where the mass-action equations balance, 
£ = 0. The action ([55]) is bilinear in the fields and 
(f)^ , so that it has a simple relation to either of its gradi- 
ents. 



E 



. dC 

94 



E 



dC 



(65) 



Therefore, the stationary point conditions, together with 
£ = 0, imply S = when evaluated on any stationary 
path. Thus the only contribution to FT(log Zq, log z^) 

must come from the boundary term Fq (log (/)3,g, log 01^^^ 

in Eq. (j49p . Since the initial distribution has support only 
on a single value Ua + rib = N, from the definition ^TE\\ 
we may write 

g-ro(iog0t^,iog0t^,) ^ gf iog(0t^0tj^_r„(iog(0t^/0t 

(66) 

in which the single-argument cumulant-generating func- 
tion corresponds to the logarithm of ipiz) in Eq. ([TI 
From Eq. dSH]) at r and large f , - 
0, and as we will see (pi and (f>\ remain nonzero, so 
log (^Jg/^jjQ^ ^ 0. Therefore, for the equilibrium dis- 
tribution Fo(log (^hg/^ao)) — > as weU. The only con- 
tribution to the generating function at time T, even with 
ZbZa = 1, is the term y -^^^('^ao'^bo) corresponding to 
the total number, which is not even dynamical! 

0, setting ZaZb — 1, we have 



-> 



In the same limit 



The resulting evaluation for the cumulant-generating 
function is then 



F^(l0g Za, log Zb) = To (log (pl^, log (/iJq 

N 



N 



log cosh i/3/L( — log cosh ^ (log z ~ (3^) 



(68) 



recovering Eq. (1^^ . We obtain the correct answer from 
a collection of surface terms, while the single-argument 
cumulant-generating functional that should have con- 
trolled dynamics dropped out. How shall we understand 
this? 
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3. Cancellation of surface terms, and the quasipotential as 
an integral of time-local log likelihoods 

The puzzle of the Gaussian evaluation of the cumulant- 
generating function is solved by taking care with the free- 
dom we have to include total derivatives in the field ac- 
tion. The particular choice that moves all fluctuation 
probabilities into the expected, single- variable cumulant- 
generating function r(logz) also sets up the action- angle 
change of variables in the next section, which will isolate 
only the dynamical observables. 

^ The property S* = 0, together with the general feature 
£ = 0, implies for a bilinear action that the sum of time- 
derivative terms equals zero independently. In particular, 
we may decompose these as 




The combination \4'\'i)b + 0l0a j is nothing more 
than the conserved number 1, while the combina- 
tion (^(t>\(i>b — 4>\4>aj /2 is precisely the observable ex- 
pected number asymmetry v. The total derivative 
— {N/2) dt log(^4'l4>\j could have been removed from the 
action to cancel the term in the final generating func- 
tion, leaving only the argument Fg (log 

causing the magnitude of rj,(log z) to originate from the 
remaining term in the action rather than from a bound- 
ary term. 

Let us summarize what has been accomplished so far: 
We have recovered the value of the single-time generat- 
ing function and the mean value of its associated distri- 
bution, but we have also derived a new inference about 
the most probable path of previous observations condi- 
tioned on those values at a time T. The fact that the 
stationary value z^t- 7^ P for r < T does not reflect the in- 
fluence of sources, or in fact any causal influence at times 
T < T, but rather the conditional probability structure 
generated by the stationary points of the field functional 
integral. This very powerful feature makes such func- 



tional integrals extremely useful, but it is also the feature 
that requires us to go through the full exercise of defining 
the generating functional of continuous-time sources, and 
then computing the Legendre transform to a Stochastic 
Effective Action, to identify the history-dependent ob- 
servations for which we are actually computing probabil- 
ities. 



4- Action-angle variables 

We next compute the same generating function using a 
set of transformed variables in which the expected num- 
ber indices n are given by elementary fields rather than 
by bilinear forms. In these variables the response field has 
the physically simple interpretation of a chemical poten- 
tial. The transformed variables have advantages and dis- 
advantages relative to the coherent-state field variables. 
The Liouville/Hamiltonian operator in the transformed 
variables will no longer be bilinear, making solutions to 
the equations of motion less obvious, even though they 
may still be found exactly. On the other hand, the kine- 
matic nature of the two-field model as a Hamiltonian 
system will be clearer. Also - although we have not 
considered fluctuations yet and have only talked about 
mean values - the transformed Hamiltonian will show 
explicitly all terms needed to compute fluctuations, in- 
cluding those that vanished in the continuous-time limit 
in coherent-state variables (p and (/)^. In the transformed 
variables it is no longer necessary to appeal to the un- 
derlying discrete-time form to compute correlation func- 
tions. 

The canonical transformation, from coherent-state 
fields to action-angle fields (including the now-familiar 
factoring out of the scale factor N), is defined by 

(/>, = e-'^'n, EE e-'^'Nv,, (70) 

for i G {a, b}. The fields rii now correspond to expecta- 
tion values from Eq. (j25p . not only as single insertions, 
but in general products of fields in the Doi-Peliti func- 
tional integral (P^. In the transformed variables, this 
integral becomes 



I 



The action in transformed variables becomes 
S ^ dt[- {dtriaUa + dtribUb) + £] 



= N dr 



- {drVaVa + drllbVb) + C , (72) 
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in which the Liouvihe/Hamiltonian has the form 

£ = fc+ (1 - e"^-"") na + (1 - e""-"") nf,, or 
C = Pf,(l-e''^-''")i., + i>a(l-e''"-'"')i.,. (73) 

We now observe an important and general relation be- 
tween the field action (I73p and the transfer-matrix terms 
in the original master equation ([3]). (Recall that the Li- 
ouville operator is defined with a minus sign relative to 
the transfer matrix.) The original index shifting opera- 
tors d/dni have been replaced by momentum fields —rji, 
and the discrete indices n^ have been replaced by field 
variables n^, whose expectation values in the functional 
integral are those of the indices n^ in the time-dependent 
density Pn„,nt,- Thus we see that the tedious task of ex- 
pressing the operator algebra and coherent-state expan- 
sion may be bypassed by a simple notational replacement, 
to arrive at the functional integral directly. The glossary 
indicating which number terms are substituted in pass- 
ing between the master equation and Liouville operator 
is given in Table. II Dl The same forms hold for the mas- 
ter equation p4l) with more general non-linear reaction 
rates, and the forms for large-deviation rates in the more 
general case are provided in App. |Dj 

The basis rotation in action-angle variables is now 
identical to that performed in Sec. IIII Al for the particle 
numbers and the chemical potentials, with i] = r]i, — r/a 
transforming as the chemical potential dual to n. The 
resulting action becomes 



S 



'7a) 



■N 



J dr {-drrjv + v^Va (1 - e^) -t- Va^b (l - e '')} 



(74) 



The first term, arising from the total derivative discussed 
at the end of the preceding subsection, is precisely the one 
needed to cancel a factor of {N/2) {rji, + rja) in the initial 
generating function ro(77ao, 7760) in Eq. (|7ip . leaving only 
the single- argument function ro(T?o) for the response field 
conjugate to v. The net effect of the boundary terms at 
time T is to set rjaf = logZa and rj^f = log zi, with the 
result that Tf (log Za, log Zb) = Tfilogz) = Tf{q) from 
Eq. (|24p . We therefore proceed to solve for the classi- 
cal stationary paths as in the preceding section, and dis- 
pense with further consideration of these extra, canceling 
terms. 

The stationary-path equations, which are the equa- 
tions of motion with respect to the dynamical system, 
in action-angle variables are 



o a/: 

drV = TT 

drV = --5-- 

or] 



(75) 



Again C = 0, but now a non-trivial relation exists be- 
tween the generating function and the stationary-path 



action, which may be written 



T f (log z) = N I dT + Cj +ro{0). (76) 

Here ly and (the implicitly present) 77 are evaluated 
over a stationary path. Of course the second factor 
£ = need not have been written, but it serves to 
emphasize the Legendre-dual relation between the Li- 
ouville/Hamiltonian operator and its stochastic "La- 
grangian" —drrju + C, which is similar to the duality 
between the generating functional and the effective ac- 
tion. 



5. The canonical versus the kinematic description, and its 
consequences for time-reversal in one- dimensional systems. 

The cumulant-generating function, evaluated as a 
stationary-path integral in Eq. (j76p . is the quasipoten- 
tial of Freidlin-Wentzell theory. For single-time fluctua- 
tions, it is a difference of Gibbs free energies, by Eq. 
With respect to the underlying dynamical system, how- 
ever, the quasipotential is an action, with the Liouville 
operator acting as its conserved Hamiltonian. The Liou- 
ville operator, however, still masks the deep simplicity of 
one-dimensional stochastic processes, which is provided 
by yet another level of description, in terms of a kinematic 
potential. The kinematic potential is the counterpart to 
the familiar potential energy in Hamiltonian dynamics, 
and it provides the most intuitive connection to finite- 
temperature instanton methods [4^. Here we show how 
the kinematic description is extracted, and use it to prove 
that, for one-dimensional systems, the most-likely path 
to arrive at any fluctuation is the time-reverse of the clas- 
sical diffusive-relaxation path from that fluctuation. This 
extension of Onsager's near-equilibrium results [l^ll^l is 
independent of the magnitude of the fluctuation or the 
form of the potential. The ability to prove it so easily in 
the general case is an example of the power of Freidlin- 
Wentzell methods in some circumstances. 

In dynamical-systems terms, v is a field with canonical 
momentum rj with respect to the Hamiltonian C. While 
77 is a canonical momentum, however, it does not play 
the role of a kinematic momentum in the stationary-path 
solutions. To express the kinematic variables, we recall 
that Da/i^b = from Sec. IIII Al or equivalently 2V = 
— tanh (/3/i/2). Correspondingly, as a purely notational 
device, for any instantaneous value of v, we may denote 



Vb 



1 = 



(77) 



so that fi is the chemical potential for which that value 
of v would result at an equilibrium. With this notation 
we may recast Eq. (j74p - as promised, now ignoring the 
surface term - as 



20 



sinh (77/2) 



cosh ( ^ (a* - j - cosh f ^7 + ^ (/^ - 



N drl -drf] V + 



cosh(^^/2) 



sinh 



V 2 



2i^ cosh 



\ 2 



(78) 



The first line of Eq. dTS]) shows us that -q- (i [ji - n) /2 
is, to leading quadratic order about zero, the term that 
creates the "kinetic energy" in the Hamiltonian C. Its 
vanishing results in the stationary solution = for 
its conjugate number field. The ry-independent part of 
the Hamiltonian, —2y/vaVbVaVb {cosh [/? (/2 — /x) /2] — 1}, 
defines the kinematic potential for the dynamical system. 
The fields rj — /3 {p, — fi) /2 follow a familiar Hamilto- 
nian phase-space dynamics in this potential, with the 
kinematic momentum vanishing at the turning points 
C = 0, and the potential vanishing only where the 
mass-action rates satisfy conditions of detailed balance. 
These total stationary points are all unstable, just as for 
problems of barrier penetration or escape in equilibrium 
Hamiltonian statistical mechanics [13] ■ A more detailed 
treatment, for problems with multiple basins of attrac- 
tion and an interesting instanton structure as a result, is 
provided in Ref. 
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The very strong consequence of the form in the first 
line, in one dimension, is that the conservation law C = 
has only two solutions at any value of j] — and 
77 = P {fl — jj,), by symmetry of the second cosh. Then, 
by antisymmetry of the gradient of the same cosh, dri' is 
equal in magnitude and opposite in sign at these two so- 
lutions. Thus, the stationary-path solution at nonzero i] 
is the time-reverse in v of the classical diffusion solution. 
Moreover, we have = /3fl — 77 at this solution, which 
together with the form in the second line of Eq. ((78|) im- 
mediately gives 



which again reproduces Eq. (f^. 

6. Langevin approximation and tfie magnitude of 
fluctuations about time- dependent solutions 



It is important that the equilibrium generating func- 
tion (IT5)) produces not only an offset equilibrium value, 
but an entire distribution corresponding to an effective 
shift in the chemical potential between the a and b states. 
The width of this distribution is identical to the fluctua- 
tion variance in v produced by the Gaussian integral with 
action (|78|) . as we now show. The easiest demonstra- 
tion is by means of the Langevin approximation, which 
is also useful to show how the classical Langevin equa- 
tion is generated from the more complete Doi-Peliti func- 
tional integral. This construction is particularly elegant 
in action-angle variables, which lead to a Langevin equa- 
tion directly for the particle numbers. The coherent-state 
variables in which the Langevin equation is usually con- 
structed [565 only offer this interpretation for fluctuations 
about classical diffusion paths, where (p^ = 1, and they 
can become quite complicated to compute about other 
backgrounds. In action-angle variables, the same inter- 
pretation is valid in for all paths, including those that 
propagate conditions about single-time fluctuation back- 
ward in time. 



i tanh ^ (»7 - /^m) 



(79) 



For these solutions we may recover the value of 
the single-time cumulant-generating function from the 
action-angle equations of motion. Vanishing C gives the 
action as an integral over its kinetic terms along the sta- 
tionary path, which no longer vanish, 

rT(logz)-ro(0) = -iV / drdrV^^ 

Jo 

/•log 2 

= ~N dT]v. (80) 



Using the expression ([75| in the last line of Eq. (|M)) , we 
then obtain 



Ft (log z) - Fo(0) = -N log cosh 1 (r, - (ip) 



log 2 



The fact that the Gaussian integral delivers the fluc- 
tuation magnitude (and more generally, the correct cor- 
relation structure) even for time-dependent solutions is 
an important improvement over an approach often taken 
with Langevin equations, which is simply to "guess" a 
fluctuation magnitude as an independent input to mod- 
els [sgI [STj . The variance produced is also the unique 
value for which the terms in the path entropies of Sec. |V] 
will cancel to produce the continuous-time hydrodynamic 
limit of Ref. |52| . 



The Langevin stochastic differential equation encodes 
exactly the same approximations as the Gaussian approx- 
imation to fluctuations in the functional integral. About 



any solutions rf 



to the stationary-point equations. 



we write general rj — 7]"^^ + rj' , v = v'^^ -\- v' , and expand S 



, (81) from Eq. (j74p to second order in primes. 
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S^S^' + N fdT h [dr + (Pfee"" + Dae-''"')] v'~]^ (Pbt^^'e"" + v^vfe-'^"') 



(82) 



In Eq. (|82]) we denote by 5^=' = / dr -drV^^v"^ + £(?7'^\ i^'^') the action of the classical stationary path. Terms 

linear in rj' and v' vanish as the condition for the stationary-point solutions to hold. 

We could complete the square in rj' in Eq. (j82|) . leading to the construction of Onsager and Machlup [111, and we 
will do this in a later section. An alternative approach, pursued here, is to perform the Hubbard- Stratonovich trans- 
formation [i^, [E^ by introducing an auxiliary field A, with a functional integral (Det [Pbt^a + '^a'^b'] ) / 'DX exp S'^"'^ 
which is just a representation of unity, in which the auxiliary field action is given by 



^Aux ^E. j dT- 



The sum of actions for the original and auxiliary fields becomes 



(83) 



S + S'^"'^ I dr 



N 



(84) 



The integral J Pry' in the original functional integral, if rotated to an imaginary contour of integration,^^ simply 
produces the functional 5-function 



/ Vr\ e V ) — d Or ^ \ Vh& ' + ' 



exp 



^^\dr 



A2 



(85) 



The last of the original functional integrals, over v' ad- 
mits only those solutions which satisfy the Langevin equa- 
tion 



dr 



y = \. 



(86) 



is given by 



{Kf) - 



2N (vbC^''^ + j^ac-""') 



A, known as the Langevin fields has the correlation func- 
tion at any two times 

(A.A.O = '^-'^°'^^^'y°^-'^"'^ (r-rO, (87) 

as a result of the Gaussian kernel of integration. 

A case of special interest, for its relation to the single- 
time distribution - at or away from equilibrium - is the 

fluctuation variance at equal times (^{v'^)^^ about a con- 
stant or long-time persistent background. We will not 
provide details about the inversion of Eq. ([55)) to express 
v' in terms of A and a retarded Green's function, which 
may be found in Ref. [s^. However, the general result is 
that for backgrounds that persist much longer than the 

(_ cl _ cl\ 
VhS^ + Vae~^ 1 , the single-time variance 

Note that r]' is always integrated along an imaginary contour, as 
the condition for stability of the Gaussian integral. This is true in 
the Onsager-Machlup construction, and it is also the imaginary 
part of Tj' that defines the quantity behaving as a momentum in 
the kinematic description. 



We will return in Sec. lIVFl to check that this result agrees 
with the variance of the single-time distribution given by 
the equilibrium generating function (j22p . 

We have given a thorough treatment of the single-time 
generating function to provide multiple points of refer- 
ence for terms that cause 0^ fields to deviate from unity, 
or rj to deviate from zero, and to interpret their effect 
on stationary points. It will now be straightforward to 
place all such terms entirely within the functional inte- 
gral rather than in boundary terms. We will do so first 
for a discrete source with an identical effect to the single- 
time generating function, and examine the future as well 
as past properties of the stationary paths. We will then 
define exact solutions for the more general case of con- 
tinuous sources, and finally study the analytic structure 
of the small-fluctuation limit, which is simply a linear 
expansion in point sources. 
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C. Generating functionals and arbitrary 
time-dependent sources 

When sources interior to functional integrals are used 
to compute the probabilities of complicated histories, the 
construction is usually done in two disconnected stages. 
First, the final-time arguments Zi are set to one, so that 
the moment-generating function il){za,Zb) becomes sim- 
ply the trace of the probability distribution at time T. 
That is, the elaborate field integrals (|49l7ip are simply 
complicated expressions for the number 1. New terms 
involving the integration variables and external fields are 
then simply inserted into the action, with the under- 
standing that these are "sources" that perturb the par- 
ticle motion. 

Here we will introduce sources systematically as part of 
the construction of the functional integral, to make clear 
their connection to the original construction of the gener- 
ating function. The construction involves modifying the 
map by which we have, up to now, simply re-interpreted 
intermediate variables z and d/ dz as operators and a. 

A generating functional is the time-evolved product of 
a sequence of generating functions produced by adding 
small continuous sources to all previous distributions. 
That is, we replace the identification (|40l) with one in 
which the abstract operators differ from variables z by 
the addition of sources. At any time r, we identify 

Zj O keeping 



—- -n- a\ 



Moreover, to separate the discretization time St from 
characteristic timescales associated with the sources, we 
will write 

qr=j{r)ST, (90) 

and take ^(t) = j^, called a current, to be a smooth 
function of r in the limit St ^ 0. Because both relax- 
ation and fluctuation effects are governed by timescales 
in At ^ 1, we may readily consider sources jr that have 
large magnitude over some range in r that is ^ 1. Cur- 
rents of this form may be constructed to approximate the 
point source jr ~ qS{T — tc) for some particular time tc- 

Finally we set values Za — I, Zb — I at time T, keeping 
only sources from nonzero j(r) in the range < t < T. 
The field integral at r = T therefore simply computes 
a trace, and all correlations are studied internally in the 
functional integral. The notation tp{za,Zb) is no longer 
needed, and we simply refer explicitly to the cumulant- 
generating functional r[j] whose argument is the function 
j- 

Both coherent-state field variables and action-angle 
variables will be of interest, and so we provide both forms 
here. Eq. (I49p is replaced by 



in which 



[^1 = / 2?0t2?0^2?42?0,e(i-*"^)*"^+(i""*^T)^''T-S,-ro(iog0+„,iog0+J^ 



Sj=N dT 



L 



(91) 



(92) 



r 



Likewise, Eq. (|71|) - removing the un-needed integration 
variable rja -\- rj^ conjugate to TV, and writing the single- 
argument generating function alone - becomes 



in which 

Sj = N J dT -~drr] v~jv + t 
with Liouville operator 

C = VbVa (1 - e^) + VaVb (l - e"'' 



(93) 



(94) 



(95) 



The upper time limit T may now be taken to infinity 
and dropped from the notation. If we consider sources 



jr at both r — >■ and t — ^ oo and which are smooth 
on the scale of the discrete sum (not a very restrictive 
condition) it follows from the stationary-path conditions 
in the presence of the source j that 



j(4>l^b-4:l^a) /2 



(96) 



is the Hamiltonian of the dynamical system in the pres- 
ence of sources. Because Cj now depends explicitly on 
time through j, Eq. ([57)1 is replaced by 



(97) 



dL _ dj 
dT dT 

We also have as boundary conditions that 7700 = 
(more generally r^ao^ = Q and 776 = independently if 



23 



we had kept both sets of integration variables), or equiv- 
alently = 1 and (pl^ = 1. To understand how these 
conditions lead to stationary-point solutions, which in- 
volves the stability structure of the functional integrals, 
it is easiest to solve some examples. The general exact 
solution, in coherent-state fields, is given in App. O 

D. General solutions 

In all the following cases, we will suppose that the ini- 
tial distribution is the equilibrium distribution (|4]) . Other 
initial conditions can easily be considered, but all con- 
verge to the equilibrium exponentially in t, and so may 
be decoupled to any desired degree from the influence of j 
at later times. We begin with a point source which recov- 
ers the single-time generating function, and then consider 
sources extended in time. 



forward direction for real-valued rj is unstable in gen- 
eral. This should not be surprising, as we have already 
seen that the evolution equations for 77 are stable going 
backward in time, as 77 propagates context from observ- 
ablcs to their most-likely prior causes. The implication 
for our stationary path is that if 77^ — and = for 
T > Tc + e, we must have r/^' = for all r > Tc + e. Hence 
Tifc-e ~ 1 ^^"^ for r < rc -f e we are back to the station- 
ary path evaluated in Sec. IIVB5I The consequence for 
the number field is that its magnitude at t = tc is just 
that of Eq. (pS)) . and that for times either before or after 
Tc , v'^^ satisfies the generalization of Eq. (IMl) to 

vf-D={vf^-v)e-\^~^^\ (100) 

The time-dependence of v'^^ is shown in Fig. 21 



Vx- V 



Point sources 




Suppose that jr qS{T ~ tc) for some time tc in the 
sense of convergence of smooth distributions with com- 
pact support and fixed area q. The exact solution in this 
case is most simply expressed in action-angle variables, 
which can then be converted to coherent-state field vari- 
ables if desired. 

From the action form ([M]) with this source, stationary 
solutions are just those of the unperturbed action except 
at TC, where rj has the discontinuity 



(98) 



as e — > 0. To understand what this implies, we return to 
the kinematic form for (|78p for the action, using r]'^^ + rj' 
as in Sec. IIVB 61 /3/i is a function of v from Eq. ([77|. 
which we indicate on the stationary path by writing /i'^'. 
For free solutions we already know that C ^ requires 
either rj^^ 
write 



FIG. 4: The time-symmetric, exponentially decaying station- 
ary path associated with single-time (5-function sources. This 
solution illustrates the embedding of the single-time equilib- 
rium large-deviations theory within the richer time-dependent 
large-deviations theory of the stochastic process. In the for- 
ward direction, classical diffusion is responsible for the decay, 
and the response field i] = 0. In the reverse direction, rj 
back-propagates the constraint that a fluctuation of magni- 
tude {vtc — ^) is observed at time re, and the dynamical- 
system representation generates the least-improbable trajec- 
tory of fluctuations that achieve that constraint. 



The generating functional for the point source has the 
value already computed. 



tions we already know that L ~ \j requires r°° 
or r;'^' = /3 (/i - /i'^i) . Therefore we may r[j] = ro(0)-iV dT drif 



cosh 



(m^'-m)) -cosh(^r,+ ^0/'-M)) 
1 — coshry') 



-iV log cosh - {rf^ - pp) 



(101) 



cosh ( ? (m"' 



sinh(r;'='-|-^(/i'^'-/i) jsinhyy'. (99) 



Our concern is with the second line of Eq. ([M)) . This 
quantity appears with positive sign in C and so describes 
divergent fluctuations for 77' integrated along a real con- 
tour. As in the case of the Hubbard-Stratonovich trans- 
formation, the convergent contour of integration for 77' is 
imaginary, where it behaves like an ordinary momentum 
variable in finite-temperature field theory (60[. 

The instability that governs the integration contour 
for fluctuations also implies that time evolution in the 



again recovering Eq. dM]). Note that although 1/^=' ^ 
at all times, the integral receives contributions only from 
T <Tc where 7/ 7^ 0. 

The effective action is numerically just that given in 
Eq. ([55)1 , because it differs from the generating functional 
by subtraction of the point source qvrc as before. How- 
ever, as a Legendre transform on a space of histories, we 
now regard it as the functional 



(102) 



whose argument i/*^' is the whole exponential history 
given by Eq. (fTUD)) with v^c fixed by Eq. (QS]). The en- 
tropy difference of Eq. (|32p is now expressed not only as 
a probability of a fluctuation of given magnitude, but of 
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the most-likely sequence of previous and subsequent con- 
figurations consistent with that observed fluctuation but 
with no other information given. 

To understand the meaning of the stochastic effective 
action as a probability on histories, it is necessary to ap- 
preciate the non-local relation between histories and the 
structure of the moment-sampling currents j which con- 
tain the minimal information to specify them. To extend 
the formulae for the effective action to more general histo- 
ries, a variety of methods are known which produce both 
non-local and exact, or local but approximate, solutions. 



2. Continuous sources; linear-response approximation and 
analytic structure 

We now return to the second-order expansion of 
Eq. (|94)) in fluctuations, from Sec. IIV B 6[ but this time 
we complete the square directly rather than through 
a Hubbard-Stratonovich transformation. Following On- 
sager [2^, [23, , we will expand about the equilibrium 
background where i''^^ = v and rf^ = 0, and will simply 
write J]' — r\. (The more general problem of expanding 
about non-classical backgrounds conditioned on sources 
could of course be considered as well.) The quadratic 
expansion of the action takes the simple form 

/•oo 

Sj — —ND drjr 
Jo 

poo 

+ / dT{{iy~D)[{~dr + l)v-'j]-'^g'>bV^}- 
Jo 

(103) 

The stationary-point equations are now linear-response 
equations. 



{-dr + 1)1] = j 
[dr + l){v - 9) = 2VaVb-n. 



(104) 



•q is the response to j through the advanced Green's func- 
tion: 

dr'e^-^'j,,, (105) 

while v—v responds through the symmetric Green's func- 
tion: 

v^-v^VaVb [ dT'e-\^-^'\jr'. (106) 

"'0 



Eq. (|106|) reproduces the exact time dependence of 
Eq. (|100|) for a point source, and gives the linear small-q 
approximation to the magnitude (j28[) . 

We now have a closed-form local expression for the 
generating functional in terms of the response field rj and 
the source j, because in these variables r[j] is just Sj 
evaluated on the stationary solution. From Eq. (|104p . 
this is 



T[j\ = -ND / drjr-- dr (2P, 
Jo ^ Jo 



vb)r]\ (107) 



with rj given by Eq. (|105p . We recognize from Eq. ([9]), 
the quantity NvaV^ as the variance of the equilibrium 
distribution and hence the susceptibility to perturbations 
in chemical potential. 

From the definition (jCllI) and use of the equations of 
motion (|104|) . the effective action is similarly expressed 
as a local functional, 



N 



1 



dr 



P. + l) {v-v)f 



2./n 2NVaVb ^ ' 



Eq. (jlOSp is the expression first due to Onsager and 
Machlup [Hi. The first line, as usual, makes explicit 
the separation of large-deviations scaling in N from the 
integral that is the rate function, which is a function only 
of fractional displacements v — v. 



E. 



The relation to Onsager's "minimum entropy 
production" property 



Eq. (|108p is a relation between an offset n — n and a 
transport current drn, which we might call v, a reaction 
velocity in appropriate units of time. Recalling Eq. ([5^ 
for the log-probability of single-time fluctuations, we may 
recognize the combination {n — n)^/ {2NDai^b) as none 
other than S'^^{n), the entropy deficit from the equilib- 
rium large-deviations principle. Immediately we would 
then recognize {n — n) / {NvaVb) as dS°^[n) /dn, and 
(n — n) V / {Nva^b) as the rate of change in the equi- 
librium entropy of quasi-equilibrated subsystems. From 
these associations we could rewrite Eq. (llOSp in the form 



'cffr, 



dr I S"="(n) + 



9S"=«(n) 1 d^S^^in) 



dn 



dn 



(II 



in which the term d'^S°^{n) / dn' gives the linear- 
response coefficients 1/ {NvaVb). 

If we chose to regard minimization of S'°*^[n] as an inte- 
gral over time of two separate criteria - the first being the 
probability of a fiuctuation to n from S'^^{n) and the sec- 
ond being a minimization over v - the minimized function 
of V given n would be the "entropy production" relative 
to a bilinear form v'^d'^S°^{n) Idr? parametrized by the 
near-eq uilibrium response coefficients. In two papers in 
1931 ^yi^, this was the result derived by Onsager as a 
consequence of microscopic reversibility and referred to 
as a "minimum entropy production" property. Onsager 
referred to the bilinear form of currents d"^ S'^'^ [n) jdr? 
as the "dissipation function" . 

Note three things: 1) It is not the "entropy produc- 
tion" per se that is minimized, but only its value rel- 
ative to the dissipation function, which from the form 
of the system entropy may be arbitrary; 2) there is no 
obvious reason to interpret the dissipation function as a 
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"constraint" on the entropy production, and thus entropy 
production is not "minimized subject to constraints" , as 
an analogy to the way the entropy is maximized subject 
to constraints in equihbrium; and 3) the presence of the 
response coefRcients in this form results from the near- 
equilibrium expansion, as is well-known. 

We see, then, how we may speak precisely about the 
range of assumptions needed to make the "production" 
of the equilibrium entropy a quantity that is informative 
about dynamics. We have required the particular form 
of the local-equilibrium approximation that comes from 
a basins-and-barriers model, so that the dependence of 
the stochastic effective action ^"^[n] on its instantaneous 
configuration variables reduces at leading order to the 
equilibrium quantity S°^{n). We have assumed small 
fluctuations in order to expand the linear velocity de- 
pendence in terms of dS'^^{ri) /dn, and to approximate 
the linear response coefficients by their equilibrium val- 
ues. Had all of these conditions not been satisfied, the 
expansion (|109p need not have been valid. Yet, within 
the more general framework of the stochastic process, we 
could readily have derived the correct alternative form 
from the exact solution, which is derived as Eq. (jCll[) 
in App. O In the more general regime of non-linear 
response and large perturbations from the equilibrium 
distribution, the relation between a source j(r) and the 
path i/(r) that it induces will generally be non-local in t. 



F. A second example: fixed dis-equilibria and an 
entropy rate for a large-deviation rate function 

We now consider a second problem, aimed at relating 
dynamics to reference static distributions. Suppose that, 
rather than assuming free relaxation immediately after a 
fiuctuation of magnitude ((28|). we ask for the probability 
of a history which remains at that value for a fixed time, 
and then freely decays. The difference of the effective 
action for this fixed-disequilibrium trajectory from that 
for a single-time fiuctuation will give the entropy rate 
difference, between the stochastic process about the non- 
equilibrium steady state and the same process acting on 
the equilibrium distribution. 

For this example, it is convenient to put the initial dis- 
tribution at time r -> — oo, to put the initial fluctuation 
at r = and hold it until re, and then to permit free 
decay after tc as before. From the constructions above, 
it is easy to define a source protocol that will produce 
this history. The source current takes the form 



steady-state condition 



dC 

dv 



J, 



(111) 



and holds it there until tc — e, at which point the second 
(5-function term takes yy'^' at r = rc+e. These results 
are easy to check from the exact equations of motion 
in action-angle coordinates, and the time-dependence of 
both Jt- and the associated i'^^ are shown in Fig. [51 





FIG. 5: The source current (dashed) and resulting stationary 
solution (solid), for a source that raises to n''' at r = 0, 
holds it at this value until t — tc, and then releases it to free 
diffusive decay. 



Because of the time-dependence of this source protocol 
J, neither C nor Cj vanishes. Indeed, because drrj = 
except at the support of the 5- functions of jr, and be- 
cause these two terms cancel in the action, the nonzero 
values of Cj and C determine F and S°^. We can check 
that these potentials evaluate to 



m = 



J dT (^-dr 



log cosh ^ — log cosh 2 ^ /^A*) 



N 



(cosh I -l) 



2j/sinh — 
2. 



(112) 



and 



= N I dT[ -drW + ^ 



N 



cosh f — 1 



2 cosh ^ cosh 2^ 



(113) 



e(Tc - r) e(T - 0) 
q 



sinh — + (cosh - — 1 
2 V 2 



+ i: [Sir - tc) + <5(r - 0)] . 



(110) 



The stationary-point background evolves rj'^^ unstably 
from its initial asymptotic value of zero at r — >■ — oo to 
value q at T = — e. The i5-function in the source then 
lowers ri'^^ to value q/2 at t = 0-|-e, which solves the 



Cj may be of either sign, while £ > always. 

We may evaluate the g <C 1 limit of Eq. (I113P for 
comparison to both the equilibrium effective action and 
the Onsager-Machlup linear approximation. Noting that 
in the linear-response regime [ly — D) ~> DaVbfli the small- 
q expansion becomes 



S' 



cffr 



NvaVbq^ 



TC- 



NVaVbq^ 



(114) 
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The probability of an initial fluctuation to non- 
equilibrium number occupancies is given by the equilib- 
rium eff'ective action, while the probability of persistence 
at that value is governed by a new term with the struc- 
ture of an entropy rate |49|. We will see in the Sec. |V] 
that this part of the effective action is a free-energy-rate 
difference, just as the equilibrium S'^^{n) of Eq. ([32l) is 
a free-energy difference. The entropy-rate difference for 
persistent dis-equilibrium may be extracted as 



d 
drc 



5^«N = Af/:.e[o+,rc] 



N cosh § - 1 
Y cosh ^ cosh ^ 



N{y/m^-y/l^) . (115) 



Eq. (I115|) is a new result. The combination of square-root 
dependence on quantities that equal the expected escape 
rates - Dti'a and Dai^t - and the squared difference of these 
in the exact entropy rate, extend to general multiparticle 
chemical reactions as shown in App. |DJ 

To make contact with the Langevin treatment in 
Sec. IIVB 6| we check that in the persistent non- 
equilibrium domain where rj = q/2, the fluctuation of 
the Langevin fleld in Eq. (|87p is given by 



(116) 



For the period of free decay after tq , the change in equi- 
librium entropy with n is 



dn 



^°"(n) = q. 



(120) 



The time-dependence of the equilibrium-form entropy 
follows from the time-dependence of n. Its gradient at 
the moment after the source perturbation is turned off is 
given by 



drn = —N {v — v) 



N sinh f 
" y cosh ^ cosh 



(121) 



Therefore, under free diffusion from a chemical-potential 
perturbation by q, the initial rate of subsystem entropy 
change is given by 



(9^S"="(n) 



— 5 

on 

N 



q sinh | 
2" cosh ^ 



^ Z 

^cosh^' 



(122) 



The decay rate in the Langevin equation (j86p evaluates 
to 



(117) 



Hence the single-time fluctuation expression ([55]) evalu- 
ates to 



(Wf > - 



N 



(118) 



in agreement with the variance obtained from Eq. (|88p for 
the distribution produced by the equilibrium generating 
function. 



1. The non- equilibrium entropy rate in relation to gradients 
of the equilibrium entropy 

How does the entropy-rate difference defined from the 
dynamical stochastic effective action relate to changes in 
the equilibrium entropy of subsystems under free decay? 
We may recast Eq. (|32|) as 



S'°tt(n) = ND{v II v) 



N 



log 



VbVa 



+ q{vb- Va) 



(119) 



At smah 9, -drS^'^in) AdS^^ln] /drc from Eq. (ITT5]) . 
Thus the stochastic-process entropy rates, and the rate 
of change of equilibrium entropies, are not equal even 
in this limit; more generally they are distinct functions 
altogether. We can check from Eq. (|115p that in the 
interval [0, tc], C has the limits £ — > j/^ as tanh {q/2) — > 1 
and £ —T' i/p as tanh {q/2) — >• —1. Thus the entropy rate 
of the stochastic process is bounded by leaving-rates from 
the two respective states, while the corresponding rate of 
entropy change in Eq. (I122p is unbounded. 

We may summarize this section as follows: It should 
not be surprising that the stochastic effective action for 
histories includes entropy-rate terms that have no simple 
relation to rates of change in the equilibrium entropy of 
states. The former measures uncertainty about rates of 
transition, while the latter measures uncertainty about 
the occupation frequencies for states. It may be that, in 
some cases, dynamics is near enough to equilibrium that 
the state occupancy at successive moments of time places 
tight constraints on the possible entropy of transitions. 
However, this is not a general result, and it is not even 
implied by the local-equilibrium approximation of the 
form produced by the double- well potential. To say more 
about the origin of the rate term in the stochastic effec- 
tive action (I113p . however, requires a separation between 
system and environment terms that the continuous-time 
two-state model does not readily provide. For that sepa- 
ration we turn to the method of maximum caliber. 
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V. ENSEMBLE OF HISTORIES II: RESOLVED 
SINGLE-PARTICLE HISTORIES; PATH 
ENTROPIES AND MAXIMUM CALIBER; 
STRUCTURAL DECOMPOSITION INTO 
SYSTEM AND ENVIRONMENT; CONNECTION 
TO THE STOCHASTIC-PROCESS ENTROPY 
RATE 

The construction of equilibrium Gibbs free energies 
from the entropy, reviewed in App. |B1 begins with an 
exphcit division between entropy terms for the system 
and its environment. The Freidhn-Wentzeh construction 
of the preceding section does not naturally provide such a 
decomposition for paths, because the contribution of the 
"environment" comes from the form of the transition-rate 
terms fc±, which are embedded in the master equation. 
To separate these two contributions in the ensemble over 
histories, we require an explicit combinatorial formula for 
properties of paths, which is defined independently of the 
probabilities given to such paths by the transition rates 
set by the environment. The method of maximum cal- 
iber, as formulated in Ref's [13, [Ml [fil] j provides such 
a decomposition. 

This section will introduce two changes of representa- 
tion. One is the consideration of histories of an individ- 
ual random walker, as explained in the introduction. The 
other, which will be more important to the ability to iso- 
late system-environment interactions, is the replacement 
of the continuous-time stochastic process of the master 
equation ([3]), by a discrete-time two-state Markov pro- 
cess, shown in Fig. [B] The continuous-time and discrete- 
time models, in appropriate limits, represent the same 
stochastic process. However, the discrete-time model has 
the feature that, in every time interval of length At, some 
transition of the particle's state must occur, even if it is 
a transition back to the same state. 

FIG. 6: Representation of the coarse-grained two-state sys- 
tem as a discrete finite-state model with forced transitions on 
every interval of length At. 



It may come as a surprise when we find, below, that the 
entropy rate in the effective action (I113P draws its form 
entirely from the probabilities of no change. When we 
separate the large-deviations formula into contributions 
from the path entropy and the environmental probabili- 
ties, other terms associated with changes will also appear. 
However, these cancel exactly in the hydrodynamic limit 
represented by Eq. (|113p . Thus we will see that, among 
the many terms that could have been used to probe the 
generating functional of the previous section, the partic- 
ular coupling to the field v that we studied was the form 
that produces the hydrodynamic limit. We return at the 
end of this section to what such a decomposition implies 
about the role of "energy dissipation" as an explanation 



for large-deviations probabilities of histories. 

In the next two sections, we will solve for the path en- 
semble first in its stationary distribution, and then per- 
turbed with sources. With appropriate choices for the 
transition parameters in the discrete model, the path en- 
semble will be constructed from the transfer matrix for 
the single-particle continuous-time model. With a partic- 
ular choice of perturbing sources in the generating func- 
tional for this distribution, we will be able to show that 
the continuous-time Langevin equation gives the Gaus- 
sian approximation to the fluctuating single-particle tra- 
jectory. 



A. Variational approach for a path ensemble in its 
stationary distribution, and connection to the 
continuous-time transfer matrix 

The implementation of maximum-caliber in Ref's. (6ll . 
[62j considers only steady trajectories displaced from the 
equilibrium. It does not consider time-dependent dis- 
placements, for which the Doi-Peliti formalism may pro- 
vide easier constructions. However, the steady-state non- 
equilibrium problem does make contact directly with the 
constant-j solutions of the preceding section. 

For the stochastic process of Fig. [5] and a finite time 
interval T, consider four observables M"^ , for a,f3 € 
{a,b}. These are the number of transitions from state 
(3 to state a between and T. If we specify no more 
than these as constraints on a path entropy, they will 
tend toward uniform distribution on a long interval T, 
irrespective of the distribution we use to initialize paths. 

The idea is to label each path with an index j , and to 
compute a path entropy 

SP-^^^^J2p^logp^^ (123) 

j 

subject to some set of appropriate constraints on prop- 
erties of the paths, such as its Af"'^ values, which will 
be denoted M"^ . It will be convenient here to divide 
the path specification into the state in which the path 
starts, labeled with index cr, and the remainder of the 
path conditioned on that starting state, indexed j \ a. 
We could think, thus, of j as a string denoting the or- 
dered states through which the trajectory passed, and 
J I CT as the string excluding its first letter. In this index- 
ing ^Pj\aPa- 

It simplifies the treatment, without omitting any re- 
sults that matter here, to exclude the distribution over 
over initial states from the variational problem, and to 
consider only variations of Pj\a- For any path j, the 

numbers M"^ count the state-transitions from state /? to 
state a along that path. Denote by M"^ the constraint 
values on maximum entropy for this path ensemble. Then 
the Lagrangian for the path-entropy maximization prob- 
lem becomes 
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M' 



Q/3 



(124) 



ik 



ik 



Lagrangians of the form (|124p produce Gibbs (expo- 
nential) distributions as their maximizing distributions. 
Therefore, in terms of the Lagrange multipliers {A"*^}, 
the maximum-entropy probabilities have the form: 



J ) Pa 



(125) 



The entropy on this distribution was given the name cal- 
iber by Jaynes , to reflect the fact that it is a logarith- 
mic measure of the width of a "tube" of microhistories 
which are typical within the distribution that produces 
macrohistories characterized by the {M"^}. 

Following Ref 's. [61), |62j] (apart from a sign convention 
for the {A"'^}) we define 



7 



Q/3 



(126) 



and choose Pa- to be the starting probability appropriate 
to whichever is the first state in history j. For this set 
of observables, the order in which transitions occur does 
not affect the weight function, though it does restrict the 
set of possible paths. 

Q is the partition function on histories, defined as the 
sum over j of the other terms in Eq. (|125p . Because 
histories are in 1-1 correspondence with monomials that 
arise from the Af th power of a 2 x 2 matrix, the partition 
function is immediately expressed as the usual trace of a 
transfer matrix, just as we could have computed working 
directly from Eq. above: 



Q = 



1 1 



ry^a ^ab 






^ M 


Pa 








^ba ^bb 




_Pb _ 



(127) 



Now we may make some simplifying assumptions with- 
out loss of generality, to bring the expression (|127l) into 
direct correspondence with the transfer matrix of the 
continuous-time model. The matrix 7 does not actu- 
ally have four independent values {7"^}, because the 
original Lagrangian (|124p did not have four indepen- 
dent constraints {M"'^}. The sum Y.afs = be- 
cause for each possible path also J2ai3 = More- 
over, for large M, it is not possible to set Af"'' 7^ 
by more than ±1, so effectively these must be chosen 
equal. Therefore two normalizations of the {7"'^} may 



^"^ Note that if we were imposing more finely resolved time- 
dependent constraints, the same might not be true, but for this 
finite set of four observables covering indefinite M , we have no 
other choice consistent with only two states. 



be chosen arbitrarily. These may be chosen so that 
^aa j^^ba ^ j., 7'''' -[-7^'' = 1, effectively choosing unit 
normalization for the partition function Q. The matrix 
[7"''] is then called a stochastic matrix. 

Next, in keeping with the omission of the initial state 
from the variational problem, suppose Pa is chosen to be 
the stationary distribution of largest eigenvalue preserved 
by whatever values of 7"'' solve the variational problem. 
Then it follows immediately - by collapse of the matrix 

[7°'^] with [ 1 1 



the right - that 



on the left, and with 



Pa Pb 



(128) 



We solve for the {7"^} by setting Eq. (|128p equal to 
Mo^fiis jt follows from our choice to normalize [7"''] to 
be a stochastic matrix that 
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M 



P0- 



(129) 



The relation of the {7"^} to the {pp} may now be 
made explicit. The assumption behind the use of a 
discrete stochastic-process model is that At is a scal- 
ing variable that may be changed in the discrete model 
while leaving important physical observables unchanged, 
though rescaling At may introducing overall renormal- 
ization constants associated with the discretization. In 
general the {7"^ } will then depend on the At but the sta- 
tionary distribution of the stochastic process should not. 
Denote the stationary distribution that is self-consistent 
with the {7"''} appropriate to the given {M"^} by 
[tt/j].^^ Then the most general stochastic matrix [7"^] 
with [tt^] as the dominant eigenvector is 



^aa 


^ab 
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+ 




(l-r-«)j 


ryba 


jbb 






1 




T'b -TTq 



(130) 

The constant R sets the frequency of transition events 
M'^b _ j^.jba ^ model ensure that the discrete stochastic 



If we had not supposed that {pa} was the stationary distribution, 
the same result would be obtained asymptotically in large M, 
through the approximation of log Q by its largest log-eigenvalue, 
denoted logA_|_. This is the approach taken in Ref's. |61|. |62|| . 
The distribution j^ir^j clearly corresponds to the number fraction 
l^i^^j of the preceding sections. 



29 



process has the same continuum hmit as the continuous- 
time model of the previous sections, we require that for 
sufficiently short At, R must converge to 



i?= (fc+ + fc_)At. 



(131) 



The form (jl30p is chosen so that under refinement or 
coarsening of the time increment, the definition (|f 3f P 
remains consistent. 

The interpretation of k± as rate constants is completed 
if we set their ratio from the fractions of time intervals 
in which the system starts, respectively, in states a or b: 



k^ + kJ 



(132) 



corresponding to Eq. ([S]). Values of k± are then chosen 
to satisfy the constraints (|129p . With these choices the 
matrix [7"^] becomes precisely the transfer matrix for 
the continuous-time one-particle problem. 

Note that the relation between the rate constants, equi- 
librium frequencies, and transition numbers may be writ- 
ten 



T 



(133) 



This form - a variant expression for either i:ak+ or Tr^fc- 
- which characterizes the equilibrium distribution, will 
arise again when we use the generating function to probe 
steady non-equilibrium states. 

From the variational property (I128P and the original 
Lagrangian formulation of the problem, we recognize that 
log Q will be the negative of a Gibbs free energy. From 
the fact that logQ = when the {7"^} are evaluated at 
the above conditions, we know that the free energy may 
be offset by a constant at the entropy-maximizing, equi- 
librium distribution. We may derive the exact relation 
by calculating the entropy of paths (the caliber) on the 
solution 

^path [|M-/^}] = - ^ TT^ log 7r„ + ^ (M"^) A"^ 
a afi 



^^„log7r„-^A7"'5log 
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■ ^ ^„ log TT, - M Y^t^pY. log '^"^ 

a 13 a 



(134) 



The first line of Eq. ()134p is simply the evaluation of the 
definition ()123p , recalling that log Q = in the normal- 
ization of . The second line is the proper extensive- form 
dependence of the entropy on the constraint variables 
which are its macroscopic arguments. The third line, in- 
terpreting the {7"'^} as transition probabilities in the 
state diagram of Fig. ©, expresses the path entropy as 



a sum of the entropy of the initial distribution, with the 
time-integral of the entropy rate of the stochastic process 
in that distribution [i^. This expression is the first in 
the paper, in which the entropy rate is given its familiar 
form from the stochastic process of a finite-state system. 

The combinatorial interpretation of the second line in 
Eq. ([T34| is immediate. Using S'°({7r"}) to denote the 
single-time entropy of the initial distribution, we may 
express 



-log 



The combinatorial entropy is the Stirling approxima- 
tion for the logs of the two independent binomial factors 
for distributing M^"- transitions a — >■ 6 among MtTq — 
j^aa _|_ j^j-ba f^Q^^i exits from state a, and similarly for 
distributing exits 6 — > a among Mtti, total exits 

from b. Note that the entropy rate term in Eq. (|134p 
will diverge logarithmically in At if a finite number of 
transitions M^°- = is fixed in a growing number of 
opportunities M. 

Finally, from the explicit expression for the entropy ei- 
ther in terms of {M"^} or the associated Lagrange mul- 
tipliers {A"'^ = — log 7"^}, we may identify the relation 
between the partition function Q and a free energy that 
leads to the relations (|128p 

- logQ = ^ X'^f^Mf - 5P""^[{m"''}] + ^°({^"}) . 

(136) 

Here each M"^ denotes the a/3-component of the set of 
M jointly satisfying 



A 



(137) 



From the developments of the single-time equilibrium, 
we may anticipate that the effective action for other con- 
figurations {M"*^} than the most-probable ones will have 
the same form as Eq. (fT36)) . with [X"'^] fixed and {M"''} 
permitted to vary independently of them. We will re- 
turn to that and to its physical and combinatorial eval- 
uation in a moment, but first we do the construction 
methodically through a generating function for the path- 
probability distribution. 



B. Generating function for steady non-equilibrium 
states, and connection to the continuous-time 
Langevin equation 

A generating function for the discrete-time path en- 
semble may be constructed as it was in the Doi-Peliti 
method. Because we will consider only steady non- 
equilibrium distributions, we may do this in a direct and 
simple way, as a modification of the underlying trans- 
fer matrix by shifts in the chemical potential. There are 
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many perturbations that could be chosen for the discrete- 
time model, but only the one equivalent to the coupling 
to the number field in Sec. IIV CI produces the same hy- 
drodynamic limit as At — >■ 0. 

Because the discrete model has four parameters {7"^} 
for transition rates, we introduce four complex variables 



(138) 



as arguments of the moment-generating function. Sup- 
pose that, like the A"'', all C"'^ scale ~ At < 1. 
The terms C''° and C"'' modify the transition currents, 
while terms C"" and C^'' modify the persistence probabil- 
ities. The counterpart to the current j of Sec. IIV C| which 
effectively modifies the chemical potentials of the states, 
is the combination — C}'^ . Therefore, to compare the 
path entropy to this perturbation in the Doi-Peliti for- 
mulation, we set C^" = C'' = 0> and consider only the 
remaining two perturbations. 

For constant sources z"" and z^**, the path-generating 
function for non-equilibrium, asymptotically-steady tran- 
sition numbers may simply be written in terms of a 
transfer matrix that has the same modification in every 
timestep, as 



cr j|cr 



y z / 
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(139) 



The starting probabilities {tTo-} in Eq. (|139p will no longer 
be those preserved by the transfer matrix at nonzero 
{C"''}, so the function rP^th f^^aa ^ ^bb^^ generally dif- 
fer from the form p36p given for — logQ. The difference 
will be associated with transient decay from {tt^} to the 
perturbed stationary state, and will scale as (T)° at large 
T. If we denote this constant as A°, the asymptotic ex- 
pression for the generating function becomes 



ppath(^aa^^6fc) = + ^^^^ K+0 



a/3 



that the off-equilibrium transition rates satisfy 



(141) 



Eq. (jl41l) is the counterpart to Eq. (|133p for the equilib- 
rium distribution. This property will be responsible for 
the cancellation of terms between the path entropy and 
external weight factors which could diverge as At ^ 0, 
and recovery of the hydrodynamic limit. 

The transition rates (|14ip equal the rate at which the 
Langevin field correlation function (I87p injects noise into 
the number- field correlation function (jSSp . Thus we con- 
firm that, for this simple example of non-interacting par- 
ticles, the Langevin equation ([55)) describes the fluctu- 
ating trajectories of single- particle states. The Langevin 
description is indirect, as v' is formally the expectation 
value of a Poisson distribution, and not a discrete single- 
particle trajectory. Following the injection of noise by 
the correlation function, it is then the classical relax- 
ation time of these fluctuations, given by Eq. (|117|) . that 
causes the single-time fluctuations ([55]) to agree with 
those in the equilibrium generating-function distribution 
from Eq. (|^^ . We thus establish the mutual consistency 
of all of the approaches used. 



C. The effective action from caliber and its 
interpretation 

From here we compute the effective action by Legendre 
transform, just as in Sec. IIIII for the single-time equilib- 
rium distribution. The result is 

= J2 A^'^M"^ - SP'''^ [{^"'^ }] + A° + log Q 



= ^A"''M""-log 



Mpa 



A" + loE 



(140) 



y A"''M"'' - log , _ , 
a \ 

AO + logQ. 



Mpb 



(142) 



The asymptotic stationary values p", p'' may be com- 
puted as functions of A -I- as before, and the functional 
relation may then be inverted to assign values to the C"^ . 
It is possible to check that only the combination — C'''' 
appears in any of the transition-number expressions, so 
we could originally have set — as a single ar- 

gument to ■0P'**'^. This construction exactly follows the 
construction of the two-argument and the one-argument 
moment-generating functions in Sec. IIIIBI 

An important secondary relation that is not merely a 
definition but results from the choice C'"' = C,"-^ = 0, is 



In the last expression we recall the decomposition (|135p 
of the stationary-state path entropy into the logarithms 
of two independent binomials. For the equilibrium tran- 
sition rates A° -> S'°({7r°}) and the expression vanishes 
as required. Recalling that S'°({7r"}) -l-log Q in Eq. (fT36)) 
is simply the maximizer of the terms written explicitly 
in Eq. (|142l) . we recover exactly the difference of Gibbs 
free energies which was the stochastic effective action in 
Eq. dsn). 

It now remains only to check that the expression (|142p 
recovers the non-divergent entropy rate computed from 
the master equation, and to assign physical meanings to 
the terms that appear. 
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Consider the limit in which M^"' = M""^ remains finite The external source term that appears in the effective 
as At — >■ and M ^ oo, and suppose all k±At ^ 1. action (|142p evaluates to 
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(143) 



The first two terms in the second line, although proportional to the expected escape rates from the two wells with 
occupancies pa, pt, actually come from the accumulation of time intervals in which the particle does not escape and 
whose likelihoods differ from unity only by k±At. This accumulation of terms would not be affected even if we made 
finite changes in M''" or M"'', as long as total M was sufhcicntly large. The other term, containing the logarithm, 
comes from the unlikely events in which sufficient energy accumulates in a fluctuation to induce a barrier crossing. 
It is simply the product of Boltzmann factors on a set of independent intervals of length At, whose total number is 

The terms in the purely combinatorial path entropy evaluate to 
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We have placed the terms in the same order as those 
in the external linear function (I143p . and the term with 
the logarithm - again arising from the events where the 
state changes - is in fact identical in the entropy and 
in Eq. (|143p . The two log terms exactly cancel in the 
subtraction that defines the effective action (|142[) . leaving 
a finite remainder, which we will see is the hydrodynamic 
entropy rate. 

It is the first two, non-divergent term, which differ be- 
tween the path entropy and the external probabilities. 
The total numbers M**" and M"'' are frequencies entering 
the binomial factors in the path entropy, and both terms 
contribute identical factors proportional to M^"" — M"-^ . 
The difference of Eq. (|143l) and Eq. (I144p gives the non- 
divergent effective action by Eq. (|142p . which goes at 
large T (ignoring the contribution of terms ~ (T) ) to 

^cffrr^^aMi __ 

— pak+ + pbk^ - 2^Jpak+pbk_ 

= [VP^-Vpi^Y- (145) 

Eq. (I145P recovers precisely the 1-particle coefficient of 
tc in Eq. (|113p . when converted to descaled time tq by 
the factor fc+ -I- fc_ . 

Thus we again find that T is the scaling variable in 
the large-deviations principle associated with indepen- 
dent path fiuctuations for a single particle. This scaling 
applies only to the extended-time partition function. It 
acts independently of the scaling with N, which is the 



same in the extended-time and single-time levels of ther- 
modynamic description. 



D. Discretization divergences and the natural scale 

As was explained in Sec. Ill El the discrete time interval 
At may be taken only down to a certain scale, before the 
approximation by a discrete two-state theory becomes in- 
appropriate. The lower limit to A< from the continuum 
double- well model will come from the diffusive relaxation 
time, for a trajectory starting at the saddle point and 
ending in whichever well is most-quickly reached. On 
timescales shorter than this, it becomes invalid to char- 
acterize escapes as elementary events counted by Mj"^ or 
Mf. 

We expect that when the discretization scale is set to 
this value - termed the natural scale of the problem fol- 
lowing Ref. |4^ - the arguments of the logarithms in 
Equations ()143ll44p become 

Atk± w 27re"'^ (146) 

The exponential waiting times in k± are not affected, but 
the factor At cancels the dimensional prefactors in k±, 
which have dimensions 1/time, and have a characteristic 
scale set by the diffusive relaxation rate. 

If we plug the approximation (|146p into the for- 
mula p43p for the environment terms in the path-space 
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Gibbs free energy, this term evaluates approximately to 

w Pak++Pbk- 

+ —P (mi - Ma) + — (Mt - Mb) • 

(147) 

We have thus arrived at a mathematical expression 
of the graphical heuristic for thinking about dynamical 
ensembles given in Fig. [3] of Sec. Ill CI The attracting 
fixed points of the underlying free-energy landscape carry 
charge- valued, extensive state variables such as the occu- 
pation frequencies pa-, pb for states. They are related 
under Legendre transform to dual, intensive entropy gra- 
dients from the environment ~ here fc+ and fc_ respec- 
tively - which determine the probabilities for each state. 
The extensive, current-valued state variables - here the 
numbers M^°' and M"^ of transition events - live on the 
saddle points of the landscape. Under Legendre trans- 
form, they should be coupled to the dual chemical poten- 
tials that determine the probability of transitions: here 

— and /i| — /i^ . The asymmetry between states and 
transitions arises because chemical potentials appear ex- 
ponentially in the leaving rates k±, and only linearly in 
the transition-state terms involving /ij. 



E. Connection to energy dissipation 



The path-space Gibbs free energy (I142p has separated 
events into categories, which we can associate with dif- 
ferent kinds of energy transfers. The events in which 
the particle exchanges wells could correctly be associ- 
ated with either absorption of energy from the bath, or 
dissipation to it, depending on the sign of the free-energy 
difference in the starting and ending well. However, these 
terms entirely cancel between the path entropy and the 
environment-mediated probability, and do not contribute 
to the entropy rate expression in Eq. (jll3p in the hydro- 
dynamic limit. This cancelling term may be ascribed to 
"entropy production" in the path ensemble which is ex- 
ported to the environment, but it does not appear in the 
large-deviations formula. Moreover, we are lucky that we 
do not need it to compute path-probabilities, because it 
appears to depend on the model resolution At, which is 
not physically meaningful. 

The terms that survive in the entropy rate, and con- 
tribute to its distinctive form (|115|) . come from all the 
moments when thermal fluctuations do not lead to es- 
capes. For all these, energy is neither absorbed nor dis- 
sipated. In summary, we acknowledge that there are 
cases |63] in which a collection of non-equilibrium transi- 
tions is constrained by consistency with the equilibrium 
distribution that is their starting or ending point. Since 
the equilibrium distribution is dictated by energy, these 
constrained transformations must also have characteriza- 
tions in terms of energy. However, for a large collection 



of non-equilibrium fluctuation theorems, energy alone is 
not serving as the primary constraint on kinetics, and 
we must look for the explanation of path probabilities in 
terms of other parameters of the stochastic process. 



VI. DISCUSSION AND CONCLUSIONS 

A. Why generating functions for large-deviations 
formulae? 



We remarked in Sec. IIV Al that generating func- 
tions provide a more natural representation for large- 
deviation formulae, and better approximation methods, 
than the discrete probability distribution, because gen- 
erating functions expand in moments of collective many- 
particle motion, rather than in single-particle hops. We 
have also seen, comparing Eq. (ITil) to Eq. (15^ . that the 
cumulant-generating function extracts the leading expo- 
nential term in from the probability distribution, bypass- 
ing sub-leading terms from normalization. But how does 
it perform this separation? We may understand that the 
isolation of leading exponentials is in fact a direct re- 
flection of the way the generating function expands in 
moments. 

In elementary statistical mechanics [TTj , we argue that 
the microcanonical and canonical ensembles produce the 
same entropy, because the probability of the most-likely 
element in a sharply peaked distribution is equal at lead- 
ing exponential order to the weight of all typical elements 
in that distribution. The two probabilities differ only by 
the width of the distribution, which is sub-leading as we 
saw in Eq. p7| . 

The moment-generating function may be understood 
as extracting, not only the probability to observe a sin- 
gle sample-fluctuation from the equilibrium distribution, 
but in fact the projection of the original distribution onto 
the entire sub- distribution within which that sample fluc- 
tuation is typical. This is the shifted binomial distribu- 
tion appearing in Eq. (|19p . The constant of proportion- 
ality between the two distributions, whose logarithm is 
the cumulant-generating function, is then precisely the 
leading exponential difference of probabilities, because 
normalization terms in the original distribution and the 
projected distribution have canceled. 



B. Operator algebras in the Doi formalism; 
two-field methods in general, and the difference 
between classical and quantum systems 

The Doi [1, Q construction of time-dependent ensem- 
bles uses a further property of generating functions which 
can lead to confusion on first exposure, but which is 
important to understand in order to assign the correct 
meaning to the operators and states used in the con- 
struction. 
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Any probability distribution may be written as a sum 
of modes, and the generating function associated with 
that probabihty distribution may hkewise be written as 
a sum of polynomials. Therefore, generating functions 
satisfy a property of superposition, similar in many re- 
spects to the superposition satisfied by states in quan- 
tum mechanics. Time evolution acts on superpositions, 
either of quantum states, or of the modes of a generating 
function, independently. Much of the time-dependence 
in either quantum mechanics or stochastic processes can 
therefore be understood from the sequencing of the time- 
evolution operators, without reference to the particular 
function-spaces in which they act. Doi's contribution was 
to distill the dependence in the order of time-evolution 
operators into an operator algebra, which has the same 
form as the algebra of the simple harmonic oscillators of 
quantum mechanics or quantum field theory. 

For this reason, Doi-Peliti methods are sometimes [l6l| 
termed "Quantum Field Theory" methods for stochas- 
tic processes. However, the operator algebra itself car- 
ries no implication of a physical property of quantum- 
superposition of distinct eigenstates of observables, and 
derives only from the linearity of time evolution. The 
physical commitment to the classical or quantum nature 
of a system comes from specifying the space of functions 
representing states and their observable properties, and 
on the inner product in that space. For further discussion 
of the many similarities implied, solely by superposition 
and causality, between classical and quantum systems, 
see Ref. [S^. It is under the correspondence of opera- 
tor algebras that the stochastic effective action and the 
quantum effective action are equivalent objects. 

The use of time-dependent sources to probe complex 
histories with the stochastic effective action does not 
seem common in reaction-diffusion applications of Doi- 
Peliti theory. We hope that such methods can come to 
be used to ask a wider range of structurally explicit ques- 
tions about non-equilibrium paths. 

As the simplest non-trivial examples, this paper has 
demonstrated new entropy-rate formulae for steady non- 
equilibrium paths, both for independent-particle random 
walks, and for stoichiometrically non-trivial chemical re- 
actions. An area for future work is to extend such for- 
mulae to study the large-deviations behavior of random 
walks on graphs or the fiuctuations of realistic chemical- 
reaction networks. 



C. The information conditions that embed 
single-time ensembles within ensembles of histories 

The equilibrium (canonical) thermodynamic ensemble 
was recovered in Sec. IIV Cl as the projection of the larger 
ensemble over histories onto a single time slice, in a re- 
gion where the histories had settled into their stationary 
distribution over both states and transitions. "Fluctua- 
tions" in the equilibrium ensemble have no dynamical 
connotation, and are simply sample values. Histories 



have both dynamical correlation - a property of a path it- 
self ~ and stochastic fluctuation that represents sampling 
from the ensemble over paths. 

Fluctuation values in the equilibrium ensemble, when 
embedded within the larger ensemble of histories, have 
the interpretation of instantaneous conditions on histo- 
ries accompanied by no other information. These infor- 
mation conditions imply distributions over histories con- 
ditioned on the single-time fluctuation, which appears as 
an independent variable. Freidlin-Wentzell theory repro- 
duces the probabilities for such single-time fluctuations 
from the integrals, over the most-probable histories, of 
conditional probabilities for each successive increment of 
change. 

The equilibrium distribution is not the only ensem- 
ble that can be recovered from an ensemble of histo- 
ries by projection onto single instants. Projections onto 
slices where paths have not settled into their stationary 
distribution result in non-equilibrium entropies at sin- 
gle times. These entropies require both charge and cur- 
rent state variables as arguments [13, El] , and they can 
be written in terms of the Wigner distribution, consid- 
ered in Ref. [s^. We may say, then that there is a hi- 
erarchy of ensembles, from the most-complete ensemble 
over histories, to single-time non-equilibrium ensembles, 
which lose the time-correlation of histories but preserve 
current-valued state variables, flnally reaching the equi- 
librium ensemble, which projects onto only the charge- 
valued state variables. By connecting state variables to 
flxed points of free-energy landscapes in Fig. [3] , and to 
Legendre duality in Eq. (I147P , we have attempted to pro- 
vide several perspectives to understand this duality. 



D. Thermodynamics beyond Zeno 

We may understand the dynamical system of Freidlin- 
Wentzell theory as doing, for thermodynamics, what 
Hamiltonian dynamics does to free mechanics from 
Zeno's paradoxes. The problem in both cases is to rep- 
resent motion as an inherent property, independent of 
position, within single instants of time. In quantum me- 
chanics this independence is manifest - traveling waves 
are distinct superpositions from standing waves - but 
it comes at the price that position and momentum are 
non-commuting observables [13, E^- The correspondence 
between two-fleld methods for stochastic processes and 
dissipative quantum systems (annotated bibliography in 
App. 1^, shows that the Doi-Peliti construction is re- 
vealing essentially the same form of inherent duality as 
the standing/traveling wave duality in quantum mechan- 
ics. The remarkable insight from Freidlin-Wentzell the- 
ory, not obvious beforehand, is that this dual momentum 
variable for classical stochastic processes should be an 
"inference fleld" , and that in reversible random walks it 
takes the form of a chemical potential. 
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E. The master equation versus maximum caliber 

Ref's. [13, HH, [G^I emphasize the potential for mas- 
ter equations to give only a coarse-grained description 
compared to maximum-caliber methods that track dis- 
tinguishable particle trajectories. For interacting parti- 
cles, this difference may be important. However, for non- 
interacting particles, the iV-particle moment-generating 
function is simply the iV*^ power of the one-particle 
moment-generating function [50|. Indeed, this relation is 
what causes the field normalizations (|53l70p to lead to an 
exact (and not merely asymptotic) factorization of scale- 
factor N from the remaining large-deviation rate func- 
tions for relative numbers, not only for the two-state sys- 
tem, but for any non-interacting particle system. We also 
see that the Doi-Peliti construction "contains" a descrip- 
tion of single-particle trajectories, from the agreement of 
the transition rates (I14ip in the discrete-time model with 
the correlation function (|87|) of the Langevin stochastic 
differential equation, whose solution is the Gaussian ap- 
proximation to discrete random walk. 

A useful further exercise would be to map the generat- 
ing functions for the master equation of Sec. |lV]and the 
discrete-time transfer matrix of Sec. |V] term-by-term, to 
understand why the latter readily yields a decomposition 
with the structure of the Gibbs free energy, while in the 
former this is hidden. In this way it might be possible to 
extend the flexibility of Doi-Peliti effective action meth- 
ods directly to maximum caliber. 



F. On the necessity of thermodynamic limits 

Our rather conservative extension of large-deviations 
calculations, from equilibrium to non-equilibrium statis- 
tical mechanics, has the advantage of starting in well- 
known territory, and taking a small step across which we 
can construct all relations between the different ensem- 
bles explicitly. It does not do justice to the generality of 
large-deviations formulae, emphasized in Ref. 0, and up 
to now we have not adequately expressed the dependence 
throughout science on the existence of thermodynamic 
descriptions. 

The ability to coarse-grain has been our most impor- 
tant tool to bring an infinitely complicated world within 
reach of analysis and comprehension. In the early years 
of statistical mechanics, when the formulations of Boltz- 
mann and Gibbs [iHl were seen as "replacing" 

classical thermodynamics, Fermi [loj was quick to em- 
phasize that thermodynamic descriptions have their own 
internal consistency and, at the appropriate level of scale, 
are self-contained with respect to empirical validation. 
We have learned over the last half-century that this point 
is true and important to a degree that Fermi could not 
have foreseen. As renormalization methods have become 
well- understood conceptually '^^ have learned that 
all descriptions are - or may as well be - thermodynamic 



descriptions [illlil.^o 

Our view of the meaning of fundamental theory has 
changed accordingly. All theories once regarded as fun- 
damental have been effective theories, and at the same 
time, from criteria of scale-dependent statistical reduc- 
tion, each has been fundamental. Whether the series 
terminates is now understood not to be critical; any level 
at which one can form a thermodynamic description is a 
valid starting point for deduction at that scale and larger 
scales of aggregation. 

Hierarchies of nested phase transitions [gI, [g^, con- 
nected by regions of large-deviations scaling, govern the 
behavior of equilibrium matter at all scales now known, 
within theories that are already understood. Recognizing 
that neither large-deviations scaling nor phase transition 
are phenomena linked inherently to equilibrium, we may 
ask how many non-equilibrium cases of symmetry break- 
ing and large-deviations scaling are responsible for com- 
plex dynamics in nature that are not yet well-understood 
in terms of collective phenomena [70j . 
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Appendix A: Related or equivalent approaches to 
large-deviations theory for non-equilibrium 
processes, and pointers to literature 

The main text brings together Doi-Peliti construc- 
tion methods, the larger framework of Freidlin-Wentzell 
theory, and Jaynes's methods of defining combina- 
torial entropies for paths. Mathematical differences 
among the methods are small; Doi-Peliti theory may be 
seen simply as a construction method for the Freidlin- 
Wentzell quasipotential, and the only important addi- 
tion from path entropies is the finer resolution it gives to 
distinguishable-particle trajectories. 

What will not be apparent in the main text is that 
several distinct literatures exist for these nearly-identical 
mathematical methods, which have been repeatedly dis- 
covered for quantum mechanics, and for discrete- and 
continuous-variable classical stochastic processes. From 



Properly they are called effective theories. The class of equilib- 
rium mechanical theories generally recognized within the con- 
ventional term "thermodynamics" are a class of effective theo- 
ries. The defining properties used in this review are, however, 
properties of the whole class, and so the terms "thermodynamic 
description" and "effective theory" could be used interchange- 
ably to refer to them. This concept of "effective" theory is also 
the source of the name for the quantum and stochastic effective 
actions. 
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the collection of different special cases, it has become 
clear [s^ that a single framework of two-field functional 
integrals subsumes all the different approaches. More- 
over, all integrals of this type share particular structural 
features, which reflect superposition or causality, inde- 
pendently of the physical substrate to which they apply. 

We summarize in order the key ideas behind two-field 
methods, functional Legendre transforms, and entropy- 
rate methods, and finally mention a parallel treatment 
within the mathematics literature that emphasizes ob- 
servables rather than states. 



1. Two-field functional integral methods 

The most widely-used (formally exact)^""^ methods in 
the physics literature to compute both deterministic and 
fluctuation properties of non-equilibrium systems are a 
class of generating-functional methods, all of which have 
expressions as two-field, coherent-state functional inte- 
grals. The earliest ofthese were the density-matrix meth- 
ods of Schwinger [14] and Keldysh [l^, |4l| to combine 
dissipation and dynamics in quantum systems. The 
Schwinger-Keldysh form was later proposed as a gen- 
eral framework to handle dissipative dynamics in clas- 
sical or quantum systems by Martin, Siggia, and Rose 
(MSR) It was observed by Doi 0, Q that the su- 

perposition principle for generating functions for multi- 
particle classical stochastic processes is similar enough to 
that for quantum density matrices that the same opera- 
tor algebra may be used to describe both, and only their 
Hilbert spaces differ. The Doi operator algebra for dis- 
crete Markov processes was later expanded as a coherent- 
state functional integral of MSR form by Peliti 01 . 
Alex Kamenev has recently emphasized [s^ that the 
causal structure originally recognized by Keldysh, and 
reflected in a characteristic "tri-diagonal" form of the 
MSR action functional, is the unifying physical commit- 
ment that all field-theoretic methods of this kind share. 

Explanations of the common elements of superposi- 
tion and operator algebras are given in the main text in 
Sec. IIV CI and the tri-diagonal form of Gaussian kernels 
and Green's functions is illustrated in App. [C] 

Among the two-field methods, the one that most 
explicitly emphasizes the large-deviations structure of 
the macroscopic approximation is the ray-theoretic, or 
eikonal-based, approach of Freidlin and Wentzell [1, [2ll . 



With this condition we exclude very widely used but intention- 
ally approximate methods such as Langevin equations or the van 
Kampen expansion [7l| . Any of the full methods cited here may 
be reduced to Langevin or van Kampen approximations by stan- 
dard methods of Gaussian integration. 

These methods, in turn, build on the earlier work of Matsub- 
ara l6Ct [721 using field-theoretic representations of the finite- 
temperature partition function to compute time-dependent cor- 
relation functions for fluctuations about thermal equilibrium. 



IstI . [ssl [73I . This approach, introduced in basic form by 
Eyring |74{ to understand chemical reaction rates, has 
been most extensively developed for escape and first- 
passage problems j28l - l36| . It refines and generalizes 
the equilibrium barrier-penetration formulae of Wentzel, 
Kramers, and Brillouin (WKB) js^]. More generally the 
Freidlin- Wentzell method is of interest to mathematicians 
for the solution of boundary-value problems for diffu- 
sion in potentials. Freidlin- Wentzell theory introduces a 
quantity known as the quasipotential, so-named because 
it is thought of as a generalization of the equilibrium ther- 
modynamic potentials'^ The quasipotential is a large- 
deviations rate function, appropriately scaled for system 
size. It may be evaluated as an action- functional integral 
along a ray or "eikonal" of the diffusion operator f2l], [s^ , 
which has the interpretation of the most probable trajec- 
tory leading to an escape or boundary value. 



2. Functional Legendre transform and stochastic 
effective actions 

Generating-functional methods for dynamical systems 
- especially emphasizing efficient approximation - have 
been extensively refined in vacuum quantum field theory 
(QFT) and to some extent in condensed-matter physics. 
They are used in these domains to extract the lead- 
ing deterministic approximation to fluctuating quantum- 
mechanical observables, a use very close to the extrac- 
tion of leading exponential dependence of probabilities in 
large-deviations theory. The functional Legendre trans- 
form of the cumulant-generating functional is known in 
QFT as the quantum effective action .221. 1^ the basis 
of background-field methods used to identify the ground 
states or vacua of field theories, and it is the starting 
point for a variety of renormalization schemes. Efficient 
and elegant graphical methods have been derived to ap- 
proximate it within the moment expansions of Gaussian 
functional integrals. 

In the Doi-Peliti construction for discrete stochas- 
tic processes, the counterpart to the quantum effec- 
tive action is the deficit in entropy for any config- 
uration from the maximum entropy attainable under 
the same constraints. This entropy deficit is precisely 
the large-deviations rate function for fiuctuations in 
macrostates T] . For closed equilibrium systems this rate 
function is simply a difference between two entropies. For 
open equilibrium systems the contribution from entropies 
of the environment leads to the expression of the rate 
function as a difference of Gibbs free energies. The latter 
case leads to the forms developed in the main text. 



"Thermodynamic potential" as used here is often a somewhat 
indeflnite reference to the classical thermodynamic entropy or 
any of its Legendre transforms. It will be shown quite precisely 
below, exactly what combination of such potentials the Freidlin- 
Wentzell quasipotential generalizes. 
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In the large-deviations limit for non-equilibrium 
stochastic processes, where probabilities are defined on 
histories rather than on instantaneous states, the rate 
function becomes the difference between a combinatorial 
entropy defined on a path, and an entropy rate from the 
Markov process that generates the path in a stationary 
environment, as we have illustrated in Sec. fVl 

We have termed this large-deviations rate function, 
counterpart to the quantum effective action in QFT, the 
stochastic effective action. The quasipotential used in 
Freidlin-Wentzell theory to compute boundary values of 
diffusion equations in potentials is an instance of the 
stochastic effective action. For a one-dimensional sys- 
tem it is the form produced by the moment-generating 
function for observations at a single time. The gen- 
eral stochastic effective action, for arbitrary path fluc- 
tuations, is computed as the self-consistent expectation 
value of the Freidlin-Wentzell field action, in which the 
Liouville operator acts as Hamiltonian. 



3. Path entropies and maximum caliber 



ern probability theory |75l - l78| . The approaches omit- 
ted in this review have in common a starting point in 
the reverse Kolmogorov equation, which emphasizes the 
time-evolution of operators corresponding to observables, 
rather than the forward-Kolmogorov or master equation 
used here, which models the time-evolution of probabil- 
ity distributions. Rigorous proofs of convergence, which 
have not been the main concern in this article, are some- 
times made easier by using the observable representation. 



Appendix B: A fluctuation-oriented summary of the 
entropy, the roles of state variables, and the 
structure expressed by constraints, in classical 
equilibrium thermodynamics 

The entropy is the state function maximized subject to 
the constraints on the system configurations. The values 
of the constraints are given by the state variables that 
are the arguments of the entropy. 



5(!7,F,{nJ), 



The approach to non-equilibrium thermodynamics 
taken by E. T. Jayncs [8] is a direct outgrowth of methods 
based on the entropy rate of a stochastic process. (Re- 
lated methods were developed for chaotic deterministic 
dynamical systems, where the entropy rate was replaced 
by the Kolmogorov-Sinai, or metric, entropy.) Probabil- 
ities are assigned to histories, and the measure on the 
space of histories, together with any constraints, then 
determines the path-space entropy. Jaynes coined the 
term caliber for the path entropy, to suggest the fiuctua- 
tion width of a tube of micro-histories about any coarse- 
grained macro-history. 

In the text, we have used a formulation of the 
maximum-caliber principle developed in Ref's. [13, Ell, 
l62j , suitable for computing the probability of steady non- 
equilibrium distributions. In cases where the transition 
probabilities (including those altered by insertions from 
generating functions) are stationary, the calculation of 
the entropy rate simplifies using the chain rule for condi- 
tional entropies [i^. The chain rule separates the path 
entropy into an equilibrium entropy on the stationary dis- 
tribution, and a conditional entropy of transitions from 
that distribution. This compact representation appears 
in Sec. WM 



Large-deviations theory within the probability 
literature 



The methods used in this paper are those common in 
the physics literature, particularly the literature from 
reaction-diffusion theory, of which Ref's. [H, [T3| are 
representative. Parallel to this physics literature is a 
very large literature on stochastic processes and large- 
deviations theory, framed within the language of mod- 



Common sources of constraint are internal energy U , vol- 
ume V , or the numbers {n^} of different kinds of particles. 

The entropy arises as the leading exponential term in 
the relative probabilities of different states of the system, 
grouped according to the values of the constraints [/, V , 
for which that entropy would be maximum, 

gS(t/,y,{n,})_ 

In this respect, the relative probability is a more funda- 
mental quantity than the normalized probability. The 
normalization is the sum or integral of terms e"^ over 
whichever states the system can take, which can depend 
on the setting. For independent systems, the relative 
probabilities multiply, so that the sum of the entropies is 
maximized at the overall-most-likely state. It is this rela- 
tion between entropy and relative probability that defines 
the macroscopic fluctuation properties of ensembles 1]. 

The way entropy governs the structure of interacting 
systems comes from the properties of its gradients. The 
manipulation of these gradients is the main enterprise of 
classical thermodynamics [Tol |. 

The equation of state is merely a definition of the gra- 
dients of the entropy in terms of observable quantities: 



5S 



dS 



6U 



dS 



6V 



dU dV ^ 
I3 6U + /3pSV ~J2l^l^^ ^'^ 



(Bl) 



In chemical thermodynamics 1//3 = fcsT corresponds to 
the temperature in energy units, p is the pressure, and 
{fj.i} are the chemical potentials. These gradients of S 
with respect to its extensive arguments are the intensive 
state variables in the thermodynamic description. 
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Eq. (|B1I) may be re-arranged to express entropy change 
in the units of any of the constraining state variables that 
are shared between system sub-components, or between 
the system and its environment. When energy is chosen 
as the unit of measure - a natural choice since it is con- 
served among nearly all forms of systems in contact - the 
result is the usual thermodynamic statement of "conser- 
vation of energy" , 

i 

In this representation pSV and {/i^ dui} are interpreted 
respectively as mechanical and chemical increments of 
work, and ksTSS labeled "heat". Eq. ((B2|) is only a 
"conservation law" in the sense that dU is a lower bound 
on the energy change required to permit a change 6S, in 
the context where volume also changes by 6V and particle 
numbers by {(5ni}. 

The expression of competition of entropy terms in the 
relative probability, when a small system is coupled to a 
much larger environment, is that we may often expand 
the entropy in the environment to linear order in the 
exchanged amounts of any conserved quantity that the 
system and environment dynamically apportion between 
them. The result for the total relative probability of an 
apportionment that leaves U and V in the system is 

^S(U,V,{ni})-flEU-0EPEV _ ^23^ 

We have supposed in the expression (|B3|) that energy 
and volume can be exchanged, but not particle number, 
and that intra-system equilibration leads to the relative 
probability that would be maximum with U and V 
as constraints. Here /3_b and pe are regarded as fixed de- 
scriptors of the environment, by supposing that it is large 
compared to the system, so that only the first derivative 
of its entropy need be known. /3e and pE therefore are 
not functions of U or V . 

The expected or stable state of the system is the one 
that maximizes the joint probability (jB3[) over U and V. 
Ordinarily this maximization is regarded to take place 
within the fluctuations of the ensemble, and values other 
than the maximizers are never used in the combination 
S {U, V, {ui}) — PeU — PePeV in classical thermodynam- 
ics. 

If we wish to use the classical theory to understand the 
effects of constraints on system states, we think of using 
/3e and pe as control variables, imposed through the in- 
teraction between the system and the environment. The 



log-probability as a function of these control variables 
then becomes ina,xu_v [S{U,V, {ui}) — (3 eU — (3 ePeV]. 
At the maximizers, /3 — I3e and p — pE- If we con- 
tinue to denominate entropy changes in units of energy, 
and if we choose signs so that the U , V extremum is a 
minimum,^* then the log-probability is proportional to 
the Gihhs Free Energy 

G{kBT,p, {nj) = min [U + pV - kBlS{U, V, {nj)] . 

(B4) 

Now p and /3 have been set equal in the system and en- 
vironment. The subscripts pe, Pe have therefore been 
dropped, and the minimizing U and V are now functions 
of p and 13. 

If we wish to express classical thermodynamic relations 
in the form that relates most directly to the fluctuation- 
origins of the entropy, we may simply work directly with 
the logarithm of the relative probability, which is 

- /3G(fcBT,p, {nj) = max [S{U, V, {nj) -/?([/ + pV)] . 

(B5) 

We will take the sign and normalization from Eq. (jBSP 
to define a standard Legendre transform of S. 

The fluctuation theorems in Sec. IIIIl for particle ex- 
change, are similar in structure to the intermediate ex- 
pression (|B3|1 for exchanges of energy or volume. —(3G 
takes the place of S, because it is assumed that energy 
and volume equilibrate on times much shorter than the 
equilibration times for particle exchange. In addition, the 
two states of the system exchange particles only with each 
other and not with the environment, so neither Gibbs free 
energy is evaluated only to linear order. 



Appendix C: Exact solution for continuous sources 
in the time-dependent generating functional 

The non-local relation between current sources and the 
non-equilibrium paths they produce makes computation 
of the functional Legendre transform technically chal- 
lenging for all but simple cases. However, the simplicity 
of the Gaussian integral in coherent-state variables makes 
it possible to understand this non-locality with exact so- 
lutions. The coherent-state variables in this appendix 
will follow the diagonalization of Sec. IIVBI 

The coherent-state field action ((92)) may be recast in 
the matrix form [sll characteristic of all two-field action 
fmictionals,^^ as 



This convention is arbitrary, of course, but it helps to reahze 
that the choice to minimize the free energy is a reUc of the notion 
from 19*^-century mechanics that energy minimization identifies 
stable states. A convention that keeps maximization of entropy 
foremost expresses directly the origin of stability in statistical 
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degeneracy, and wc will use such a convention throughout the 
paper. 

25 Pqj. relations among these, see the literature reviewr in App. [X] 



Sj = N I dr 
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In the second line, cto designates the 2x2 identity matrix 
and the other cr, are the Pauli matrices (Ml 



CTi 



a'2 = 



-1 



(C2) 



The solution to the stationary point conditions, and for 
the effective action more generally, follows closely what 
has already been done for the single-time generating func- 
tion. The major difference is that we must replace the 
solution of scalar differential equations with the time- 
ordered matrix solution of a linear differential equation 
expressed in terms of the matrix in Eq. (|C1|) . The 2x2 
matrix kernel, which depends on parameters of the po- 
tential and on j, will be denoted by 



j] 



1 



1 



(C3) 



The expression of the boundary conditions at time T 
(here again T will be assumed finite but large) may be 
written 



= 72 



(C4) 



and in terms of these, the general solution for the ip'^ fields 
propagates this constraint backward in time, as 



(C5) 



Here denotes the inverse time-ordering operator 

which arranges terms in the exponential from right to 
left in order of decreasing time. The solution (|C5[) is the 
direct generalization of the solutions (|58I59|) . 

In the same manner, for continuous sources (no 6- 
functions, so we do not have to worry about exponential 
tails) jrKO = 0, and the equilibrium state as the initial 
distribution, the lower boundary condition for the 4> fields 
is 



^ V2 



29 1 



(C6) 



and the general time-dependent solution is 



V2 



29 1 



7— le^/o"''^'^^'^-!. 



(C7) 

Eq. (|C7I) generalizes the constant solution (pOI) due to 
nonzero jr . 

The constraint of total number fixes the normalization 
of <3?o as before, through the relation 



(at any t) 




(C8) 



corresponding to Eq. ((6T|) . The general time-dependent 
number expectation Ut is immediately seen to satisfy 
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The functional form in the first two lines of Eq. (|C9p di- 
rectly generalizes Eq. (|100l) for the point source. The last 
two lines are the functional equivalent, with a variational 
derivative replacing the partial derivative, of the single- 
time relation (|25|) . And indeed, by exactly the evaluation 
leading to Eq. , we find that the cumulant-generating 
functional is given by 



r[j] = -iviog 



2v 1 



T- 



(CIO) 

We now have an exact expression for the effective ac- 
tion which generalizes the single-time expression (l33t . 
Drawing Vr — Ur/N from Eq. (|C9p . the stochastic ef- 
fective action is 



5" 



cffr, 



N 



drjrVr + r[j] . 



(Cll) 



Eq. (jClip is the promised functional which cor- 
rectly generalizes the notion of an entropy difference 
from Eq. (l32t to a space of histories. It represents a 
large-deviations principle for the stochastic process from 

Eq. with a rate function drjri^T+^ij] /N which is 
strictly TV- independent for Gaussian fluctuations. In this 
sense the linear reaction is the analog of the Gaussian 
distribution, and its large-deviations principle is simply 
the central limit theorem for a system with finite vari- 
ance. The residual functional determinant for fluctua- 
tions about the mean value - though it will not be com- 
puted here - would introduce sub-extensive corrections in 
iV just like those of the residual entropy in Eq. (IT7|) . (See 
Ref's [H] Ch. 16, di], and m Ch. 7 for good treatments 
of such calculations). 



Appendix D: Formulae for multi-particle reactions; 
equivalently, the extension from random walks from 
graphs to directed hypergraphs 

Independent random walks between wells, by a par- 
ticles of a single type, may be modeled in the discrete- 
state approximation by diffusion on an ordinary graph. If 



the the mass-action rates satisfy detailed balance in the 
steady state, the graph may be considered undirected; 
otherwise it is directed. Chemical reactions in which 
multiple particles jointly move through a transition state 
from reactants to products generalize this relation from 
graphs to hypergraphs. A graph need not be directed in 
the case of detailed balance, because the nodes at either 
end of a link are naturally complementary. Hypergraphs, 
in contrast, are defined to permit arbitrary collections of 
nodes as the "boundary" of a hyper-edge. We therefore 
require directed hyper-edges to model chemistry, which 
distinguish a subset of nodes as inputs and a second sub- 
set as outputs. 

The surprising formula (IllSp for the probability rate 
of persistent non-equilibrium paths may immediately be 
extended to arbitrary networks of chemical reactions, 
hence, from graphs to directed hypergraphs. Global net- 
work topology will not be considered here, but the rate 
formulae for arbitrary multiparticle reactions at steady- 
state dis-equilibria will be given. 

Consider a general collection of bi-directional reactions 
indexed by their reactant and product sides as r and p. 
Individual species participating in the reaction generalize 
the wells a and b of the two-state system, to a many- 
well problem where indices Ui and bj are drawn from a 
common set of species. The reaction formula is 



ba 



Pbi 



(Dl) 



To simplify the notation, rather than writing stoichio- 
metric coefficients other than unity, it will be assumed 
that the coefficients or bj can repeatedly take the same 
values as needed. The order in which the indices ab are 
written defines the relation between the sign of the cur- 
rent and the changes of concentrations, so that ab and 
ba denote the same reaction but with opposite sign con- 
ventions for the current. When only the pair matters, 
without respect to sign, the pair-index will be denoted 
by (ab). 

The complexes of reactants and products behave in 
the transition state as a single "particle", for which the 
one-particle free energy (or chemical potential) will be 



denoted 



(ab) 



The half-reaction rate constants are then 
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defined from the transition-state and single-species one- 
particle chemical potentials as 

= (D2) 

The forward and backward half-reaction rates associated 
with the reaction (|D1I) , with the simplified dilute-solution 
assumption that activities are proportional to concentra- 
tions, are then given by 

m 
n 

Tab = fcabjjnf,.. 

i=i 

= e-K''r-E;.iA's). (D3) 

In the succeeding lines of each rate law, dropping the 
superscript 1 on the species chemical potentials reflects 
the absorption of number factors to form the n-particle 
chemical potentials, as in Eq. (|10p . 



For the general network, which may have multiple char- 
acteristic timescales in different reactions, it is less con- 
venient to find an overall non-dimensionalization of time 
than to simply work in real-time coordinates, so we will 
write all integrals with measure dt. 

If, as for the double-well, sources are coupled only to 
(arbitrary polynomials in) the rifc, the Uk equation of mo- 
tion will continue to depend explicitly only on the 77 vari- 
ables. Hence, the response to the sources will be through 
these variables. The stationary-point equations there- 
fore continue to determine quantities such as the value of 
C appearing in the steady non-equilibrium effective ac- 
tion. From these, the forms of sources needed to produce 
such backgrounds can then be inferred. The equation for 
Uk is 




Again, the sum is symmetric under exchange of a and 6, 
so wherever d/drjk acts, it acts equally and with oppo- 
site sign in two terms. Cancellation of all such pairs of 
terms corresponds to detailed balance, except that the 77 
variables in the exponential can take nonzero values. 



Eq. (jD7p can be solved with dtrik = for any configu- 
ration of the {rife} by setting 



a. Master equation, Liouville operator, and 
Freidlm- Wentzell action 

The master equation for the stochastic reaction net- 
work immediately generalizes Eq. (p4)) to 

ah 

(D4) 

Note that the reaction order matters in this expression, 
so that the same exponential of shift operators occurs 
in two terms but with opposing sign, acting on different 
products of the species numbers Uq . or n;,^. . 

The construction of the field functional integral pro- 
ceeds just as for the linear reaction. Skipping directly to 
action-angle variables, the Liouville operator generalizing 
Eq. ([75)) becomes 

C = Y.h,\{ na. (1 - e^"- . \ . (D5) 

ba i=l ^ ' 

The corresponding action functional generalizing 
Eq. (fMl) (and dropping surface terms associated with 
total number) is then 

5 - Jdti^-J2 9tmnk + . (D6) 



Vk = ^Pf^k = 2 (/S^fe + log Tik) . (D8) 



A sequence of evaluations identical to those leading to 
Eq. (|113p then produces an equivalent form for the set 
of many-particle reactions, now expressed as a sum over 
reactions only (so unordered pairs (ab)) 




Eq. (|D9I) generalizes Eq. (|115|) for the single linear re- 
action. Other terms in the effective action vanish on 
a constant-n history, and so this is the expression from 
Freidlin- Wentzell theory that governs fluctuations in the 
manner of an entropy-rate difference. A topic for future 
work is to extend the Jaynes path-entropy methods to 
many-particle reactions, to determine whether combina- 
torial and environment entropy rates may be isolated for 
these as for the double-well. 
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